-- Logs begin at Fri 2022-08-26 10:26:48 UTC, end at Fri 2022-08-26 11:25:07 UTC. -- Aug 26 10:50:35.160684 dv-cinder-852369-2-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Aug 26 10:50:35.169878 dv-cinder-852369-2-iscsi devstack@keystone.service[81409]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Aug 26 10:50:35.171334 dv-cinder-852369-2-iscsi devstack@keystone.service[81409]: *** Starting uWSGI 2.0.18-debian (64bit) on [Fri Aug 26 10:50:35 2022] *** Aug 26 10:50:35.171334 dv-cinder-852369-2-iscsi devstack@keystone.service[81409]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Aug 26 10:50:35.171334 dv-cinder-852369-2-iscsi devstack@keystone.service[81409]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Aug 26 10:50:35.171334 dv-cinder-852369-2-iscsi devstack@keystone.service[81409]: nodename: dv-cinder-852369-2-iscsi Aug 26 10:50:35.171334 dv-cinder-852369-2-iscsi devstack@keystone.service[81409]: machine: x86_64 Aug 26 10:50:35.171334 dv-cinder-852369-2-iscsi devstack@keystone.service[81409]: clock source: unix Aug 26 10:50:35.171334 dv-cinder-852369-2-iscsi devstack@keystone.service[81409]: pcre jit disabled Aug 26 10:50:35.171334 dv-cinder-852369-2-iscsi devstack@keystone.service[81409]: detected number of CPU cores: 4 Aug 26 10:50:35.171334 dv-cinder-852369-2-iscsi devstack@keystone.service[81409]: current working directory: / Aug 26 10:50:35.171334 dv-cinder-852369-2-iscsi devstack@keystone.service[81409]: detected binary path: /usr/bin/uwsgi-core Aug 26 10:50:35.171334 dv-cinder-852369-2-iscsi devstack@keystone.service[81409]: your processes number limit is 47680 Aug 26 10:50:35.171334 dv-cinder-852369-2-iscsi devstack@keystone.service[81409]: your memory page size is 4096 bytes Aug 26 10:50:35.171334 dv-cinder-852369-2-iscsi devstack@keystone.service[81409]: detected max file descriptor number: 2048 Aug 26 10:50:35.172001 dv-cinder-852369-2-iscsi devstack@keystone.service[81409]: lock engine: pthread robust mutexes Aug 26 10:50:35.172001 dv-cinder-852369-2-iscsi devstack@keystone.service[81409]: thunder lock: enabled Aug 26 10:50:35.172001 dv-cinder-852369-2-iscsi devstack@keystone.service[81409]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Aug 26 10:50:35.172001 dv-cinder-852369-2-iscsi devstack@keystone.service[81409]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Aug 26 10:50:35.211601 dv-cinder-852369-2-iscsi devstack@keystone.service[81409]: Python main interpreter initialized at 0x55b4db9489c0 Aug 26 10:50:35.211601 dv-cinder-852369-2-iscsi devstack@keystone.service[81409]: python threads support enabled Aug 26 10:50:35.211601 dv-cinder-852369-2-iscsi devstack@keystone.service[81409]: your server socket listen backlog is limited to 100 connections Aug 26 10:50:35.211601 dv-cinder-852369-2-iscsi devstack@keystone.service[81409]: your mercy for graceful operations on workers is 90 seconds Aug 26 10:50:35.212039 dv-cinder-852369-2-iscsi devstack@keystone.service[81409]: mapped 403077 bytes (393 KB) for 2 cores Aug 26 10:50:35.212039 dv-cinder-852369-2-iscsi devstack@keystone.service[81409]: *** Operational MODE: preforking *** Aug 26 10:50:35.212408 dv-cinder-852369-2-iscsi devstack@keystone.service[81409]: *** uWSGI is running in multiple interpreter mode *** Aug 26 10:50:35.212408 dv-cinder-852369-2-iscsi devstack@keystone.service[81409]: spawned uWSGI master process (pid: 81409) Aug 26 10:50:35.212619 dv-cinder-852369-2-iscsi systemd[1]: Started Devstack devstack@keystone.service. Aug 26 10:50:35.213372 dv-cinder-852369-2-iscsi devstack@keystone.service[81409]: spawned uWSGI worker 1 (pid: 81423, cores: 1) Aug 26 10:50:35.213660 dv-cinder-852369-2-iscsi devstack@keystone.service[81409]: spawned uWSGI worker 2 (pid: 81424, cores: 1) Aug 26 10:50:35.213755 dv-cinder-852369-2-iscsi devstack@keystone.service[81409]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Aug 26 10:50:39.553699 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 26 10:50:39.553699 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: from cryptography.utils import int_from_bytes Aug 26 10:50:39.554536 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 26 10:50:39.554536 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: from cryptography.utils import int_from_bytes Aug 26 10:50:39.554988 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 26 10:50:39.554988 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: from cryptography.utils import int_from_bytes Aug 26 10:50:39.555096 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 26 10:50:39.555096 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: from cryptography.utils import int_from_bytes Aug 26 10:50:39.604946 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Aug 26 10:50:39.605140 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Aug 26 10:50:39.605319 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] command line args: [] {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Aug 26 10:50:39.605520 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Aug 26 10:50:39.605601 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Aug 26 10:50:39.605717 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Aug 26 10:50:39.605806 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] ================================================================================ {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Aug 26 10:50:39.605912 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] command line args: [] {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Aug 26 10:50:39.606011 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] admin_token = **** {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 10:50:39.606209 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Aug 26 10:50:39.606287 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] config_dir = [] {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 10:50:39.606393 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] ================================================================================ {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Aug 26 10:50:39.606478 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 10:50:39.606602 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] config_source = [] {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 10:50:39.606676 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] admin_token = **** {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 10:50:39.606792 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] debug = True {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 10:50:39.606875 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] config_dir = [] {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 10:50:39.606996 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 10:50:39.607287 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 10:50:39.607287 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] config_source = [] {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 10:50:39.607732 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 10:50:39.607832 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] debug = True {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 10:50:39.607933 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 10:50:39.608034 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 10:50:39.608161 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 10:50:39.608263 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 10:50:39.608263 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 10:50:39.608448 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 10:50:39.608550 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 10:50:39.608649 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] list_limit = None {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 10:50:39.608649 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] log_config_append = None {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 10:50:39.608820 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 10:50:39.608924 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 10:50:39.609025 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] list_limit = None {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 10:50:39.609124 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] log_dir = None {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 10:50:39.609217 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] log_config_append = None {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 10:50:39.609309 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] log_file = None {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 10:50:39.609408 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 10:50:39.609508 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 10:50:39.609640 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] log_dir = None {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 10:50:39.609640 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] log_file = None {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 10:50:39.609823 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 10:50:39.609904 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 10:50:39.609994 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 10:50:39.610077 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 10:50:39.610200 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 10:50:39.610282 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 10:50:39.610404 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 10:50:39.610477 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 10:50:39.610592 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 10:50:39.610678 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 10:50:39.610754 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 10:50:39.610874 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 10:50:39.610953 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 10:50:39.611120 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 10:50:39.611206 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 10:50:39.611363 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 10:50:39.611480 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 10:50:39.611596 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 10:50:39.611670 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 10:50:39.611828 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 10:50:39.611943 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 10:50:39.612065 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 10:50:39.612209 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 10:50:39.612334 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 10:50:39.612458 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 10:50:39.612572 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 10:50:39.612682 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 10:50:39.612805 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 10:50:39.613593 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.40/identity {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 10:50:39.613779 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] publish_errors = False {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 10:50:39.613894 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 10:50:39.614100 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 10:50:39.614339 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 10:50:39.614511 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.40/identity {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 10:50:39.614635 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 10:50:39.614759 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] publish_errors = False {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 10:50:39.614877 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 10:50:39.614995 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 10:50:39.615112 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 10:50:39.615231 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 10:50:39.615363 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] standard_threads = False {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 10:50:39.615484 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 10:50:39.615602 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 10:50:39.615715 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 10:50:39.615827 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 10:50:39.615946 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 10:50:39.616059 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 10:50:39.616059 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] use_journal = False {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 10:50:39.616273 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] standard_threads = False {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 10:50:39.616385 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] use_json = False {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 10:50:39.616503 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 10:50:39.616684 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] use_stderr = False {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 10:50:39.616807 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 10:50:39.616920 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] use_syslog = False {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 10:50:39.617030 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 10:50:39.617147 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 10:50:39.617259 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] use_journal = False {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 10:50:39.617378 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.617491 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] use_json = False {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 10:50:39.617491 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] use_stderr = False {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 10:50:39.617698 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.617809 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] use_syslog = False {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 10:50:39.617920 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.618065 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 10:50:39.618179 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.618290 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.618401 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.618512 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.618629 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.618753 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.618867 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.618984 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.619095 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.619206 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.619317 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.619436 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.619548 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.619665 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.619776 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.619888 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.620004 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.620124 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.620242 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.620353 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.620470 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.620582 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.620692 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.620813 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.620937 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.621056 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.621167 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.621278 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.621388 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.621510 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.621622 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.621738 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.621850 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.621961 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.622087 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.622279 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.622394 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.622505 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.622626 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.622739 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.622858 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.622971 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.623083 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.623200 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.623311 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.623422 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.623544 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.623663 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.623780 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.623892 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.624003 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.624122 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.624245 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.624358 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.624469 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.624585 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.624695 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.624807 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.624925 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.625037 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.625148 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.625265 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.625376 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.625493 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.625612 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.625724 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.625834 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.625945 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.626092 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.626207 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.626325 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.626437 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.626548 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.626665 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.626776 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.626893 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.627012 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.627124 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.627236 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.627347 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.627463 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.627575 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.627759 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.627873 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.627990 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.628108 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.628219 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.628331 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.628456 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.628575 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.629275 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.629388 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.629500 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.629611 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.629731 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.629845 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.629956 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.630115 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.630226 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.630332 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.630437 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.630537 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.630636 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.630729 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.630828 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.630921 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.631021 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.631124 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.631219 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.631358 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.631481 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.631605 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.631718 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.631835 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.631954 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.632066 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.632184 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.632307 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.632419 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.632522 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.632639 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.632757 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.632859 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.632978 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.633092 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.633207 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.633320 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.633440 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.633555 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.633667 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.633778 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.633899 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.634029 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.634136 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.634257 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.634370 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.634472 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.634591 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.634704 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.634816 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.634928 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.635048 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.635160 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.635280 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.635392 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.635503 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.635615 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.635726 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.635844 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.635964 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.636077 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.636189 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.636300 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.636418 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.636531 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.636652 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.636771 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.636890 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.637002 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.637114 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.637233 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.637353 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.637467 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.637578 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.637691 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.637802 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.637914 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.638062 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.638178 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.638291 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.638403 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.638514 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.638625 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.638746 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.638859 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.638977 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.639089 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.639208 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.639326 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.639513 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.639628 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.639740 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.639853 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.639965 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.640085 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.640208 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.640344 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.640517 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.640657 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.640770 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.640883 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.640995 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.641118 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.641232 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.641344 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.641456 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.641567 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.641680 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.641799 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.641914 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.642046 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.642161 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.642273 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.642384 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.642384 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.642609 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.642730 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.642851 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.642851 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] auth.external = None {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.643075 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.643215 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.643340 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.643340 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.643562 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.643676 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] auth.external = None {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.643788 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.643909 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.644031 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] auth.password = None {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.644031 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] auth.token = None {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.644298 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.644412 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.644531 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.644644 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.644764 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] auth.password = None {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.644885 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.644999 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] auth.token = None {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.644999 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.645210 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.645323 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.645435 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.645556 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.645556 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.645768 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.645881 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.646006 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.646122 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.646247 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] credential.caching = True {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.646362 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.646474 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.646586 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] credential.caching = True {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.646698 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.646819 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.646945 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.647059 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.647173 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.647285 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.647404 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.647518 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.647638 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.647752 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.647865 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.647978 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.648090 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.648209 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.648335 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.648453 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.648566 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.648679 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.648791 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.648904 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.649025 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.649139 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.649253 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.649365 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.649477 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.649589 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.649710 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.649823 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.649936 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] federation.caching = True {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.650101 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.650214 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.650328 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] federation.caching = True {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.650452 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.650567 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.650686 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.650799 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.650911 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.651023 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.651147 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.651262 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.651375 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.651488 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.651600 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.651721 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.651836 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.651954 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.652068 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.652181 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.652294 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.652415 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.652530 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.652647 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.652760 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] identity.caching = True {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.652882 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.652995 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.653182 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] identity.caching = True {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.653298 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.653413 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.653525 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.653638 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.653750 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.653871 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.653998 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.654112 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.654225 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.654337 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.654450 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.654574 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.654697 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.654811 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.654923 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.655036 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.655148 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.655268 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.655382 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.655495 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.655608 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.655720 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.655833 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.655953 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.656067 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.656179 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.656291 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.656410 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.656523 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.656645 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.656759 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.656872 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.656986 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.657097 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.657210 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.657330 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.657444 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.657556 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.657669 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.657781 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.657894 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.658072 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.658188 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.658301 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.658414 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.658527 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.658640 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.658760 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.658874 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.658987 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.659099 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.659212 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.659324 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.659444 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.659559 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.659672 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.659786 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.659899 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.660011 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.660131 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.660245 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.660358 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.660471 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.660583 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.660583 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.660805 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.660805 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.661017 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.661017 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.661228 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.661348 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.661463 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.661576 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.661689 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.661802 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.661915 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.662050 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.662167 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.662279 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.662392 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.662505 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.662618 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.662743 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.662859 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.662972 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.663085 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.663197 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.663310 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.663430 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.663544 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.663657 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.663769 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.663883 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.663996 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.664116 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.664230 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.664343 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.664456 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.664569 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.664688 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] ldap.user = None {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.664809 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] ldap.user = None {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.664923 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.665037 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.665149 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.665262 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.665375 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.665496 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.665616 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.665730 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.665842 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.665956 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.666113 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.666238 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.666353 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.666466 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.666579 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.666691 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.666803 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.666924 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.667039 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.667152 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.667265 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.667384 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.667497 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.667622 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.667737 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.667850 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.667963 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.668075 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.668189 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.668309 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.668424 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.668537 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.668650 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.668763 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.668875 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.668996 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.669110 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.669224 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.669337 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.669450 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.669563 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.669683 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.669797 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.669910 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.670035 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.670149 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.670264 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.670388 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.670503 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.670616 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.670729 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.670841 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.670953 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.671074 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.671188 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.671302 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.671414 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.671527 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.671639 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.671760 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.671874 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.671986 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.672099 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.672212 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.672326 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.672447 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.672561 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.672677 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.672790 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.672902 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.673015 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.673135 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.673250 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] resource.caching = True {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.673364 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] resource.caching = True {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.673478 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.673591 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.673703 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.673824 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.673952 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.674142 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.674285 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.674401 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.674514 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.674639 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.674754 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.674874 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.674987 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.675100 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.675284 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.675401 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.675994 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.676109 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.676222 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] role.caching = True {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.676355 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] role.caching = True {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.676520 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] role.driver = sql {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.676637 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] role.driver = sql {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.676751 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.676863 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.676976 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.677089 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.677210 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.677325 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.677438 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.677553 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.677666 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.677779 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.677900 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.678028 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.678144 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.678256 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.678369 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.678482 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.678608 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.678722 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.678835 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.678948 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.679060 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.679172 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.679293 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.679407 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.679520 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.679634 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.679747 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.679860 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.679971 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.680084 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.680197 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.680310 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.680422 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.680535 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.680656 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.680772 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.680885 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.680998 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.681111 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.681231 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.681346 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.681459 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.681572 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.681685 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.681799 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.681919 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.682140 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.682256 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.682369 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.682481 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.682597 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.682728 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.682846 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.682959 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.683072 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.683186 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.683298 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.683420 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.683534 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.683647 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.683760 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.683875 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.683987 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.684112 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.684227 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.684340 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.684453 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] token.caching = True {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.684566 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] token.caching = True {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.684679 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.684801 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.684916 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.685029 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.685142 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.685262 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.685375 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.685545 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.685661 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.685774 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.685887 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.686017 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.686132 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.686257 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.686371 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.686484 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.686596 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.686709 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.686821 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.686942 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.687057 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.687170 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.687283 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.687395 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.687507 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.687628 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.687742 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.687854 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.687966 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.688089 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.688215 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.688337 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.688451 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.688565 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.688686 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] database.connection = **** {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.688800 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] database.connection = **** {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.688912 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.689025 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.689139 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.689252 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.689374 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.689488 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.689601 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.689714 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.689827 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.689946 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.690088 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.690214 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.690328 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.690441 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.690554 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.690667 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.690788 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.690903 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.691015 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.691728 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.691915 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.692040 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.692164 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.692281 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.692395 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.692509 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.692622 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.692738 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.692861 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.692976 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.693124 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.693240 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.693354 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.693467 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.693593 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.693709 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 10:50:39.693821 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Aug 26 10:50:39.693934 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Aug 26 10:50:39.722981 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81424) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 26 10:50:39.723140 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81423) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 26 10:50:39.723220 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81424) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 26 10:50:39.723357 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81423) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 26 10:50:39.723439 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81424) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 26 10:50:39.723583 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81423) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 26 10:50:39.731905 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81424) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 26 10:50:39.732083 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81423) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 26 10:50:39.761600 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81423) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 26 10:50:39.761750 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81424) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 26 10:50:39.761835 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81423) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 26 10:50:39.761996 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81424) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 26 10:50:39.815875 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81423) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 26 10:50:39.816321 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81424) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 26 10:50:39.865689 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81423) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 26 10:50:39.865920 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81423) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 26 10:50:39.866038 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81424) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 26 10:50:39.866272 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81424) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 26 10:50:40.022681 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81423) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 26 10:50:40.022917 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81423) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 26 10:50:40.023031 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81424) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 26 10:50:40.023140 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81423) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 26 10:50:40.023226 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81424) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 26 10:50:40.023365 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81423) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 26 10:50:40.023454 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81424) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 26 10:50:40.023584 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81424) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 26 10:50:40.023666 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81423) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 26 10:50:40.023801 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81423) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 26 10:50:40.023879 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81424) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 26 10:50:40.024013 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81424) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 26 10:50:40.047509 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81423) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 26 10:50:40.047681 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81424) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 26 10:50:40.047772 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81423) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 26 10:50:40.047953 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81423) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 26 10:50:40.048036 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81424) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 26 10:50:40.048182 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81423) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 26 10:50:40.048265 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81424) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 26 10:50:40.048401 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81424) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 26 10:50:40.048480 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81423) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 26 10:50:40.048620 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81423) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 26 10:50:40.048698 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81424) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 26 10:50:40.048830 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81424) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 26 10:50:40.048910 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81423) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 26 10:50:40.049042 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81424) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 26 10:50:40.060565 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81423) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 26 10:50:40.060913 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81424) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 26 10:50:40.073577 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81423) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 26 10:50:40.073764 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81423) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 26 10:50:40.073851 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81424) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 26 10:50:40.074062 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81424) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 26 10:50:40.074145 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81423) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 26 10:50:40.074290 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81423) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 26 10:50:40.074371 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81424) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 26 10:50:40.074513 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81424) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 26 10:50:40.074594 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81423) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 26 10:50:40.074730 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81424) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 26 10:50:40.074812 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81423) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 26 10:50:40.074946 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81423) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 26 10:50:40.075028 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81424) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 26 10:50:40.075155 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81424) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 26 10:50:40.092083 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81423) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 10:50:40.092295 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81423) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 26 10:50:40.092495 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81424) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 10:50:40.092645 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81423) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 10:50:40.092725 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81424) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 26 10:50:40.092851 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81423) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 26 10:50:40.093039 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81424) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 10:50:40.093120 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81423) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 10:50:40.093242 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81424) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 26 10:50:40.093394 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81423) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 10:50:40.093522 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81424) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 10:50:40.093790 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81424) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 10:50:40.093911 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81423) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 10:50:40.094187 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81423) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 10:50:40.094465 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81424) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 10:50:40.094733 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81424) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 10:50:40.104149 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81423) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 10:50:40.104434 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81423) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 10:50:40.104711 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81423) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 10:50:40.104997 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81423) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 10:50:40.105122 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81424) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 10:50:40.105361 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81424) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 10:50:40.105638 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81424) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 10:50:40.105927 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81424) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 10:50:40.112279 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81423) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 26 10:50:40.113817 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81424) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 26 10:50:40.115812 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81423) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 26 10:50:40.116104 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81423) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 10:50:40.116394 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81423) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 10:50:40.116684 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81423) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 10:50:40.116766 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81424) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 26 10:50:40.116973 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81423) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 10:50:40.117055 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81424) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 10:50:40.117242 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81423) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 10:50:40.117323 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81424) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 10:50:40.117519 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81423) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 10:50:40.117601 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81424) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 10:50:40.117798 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81423) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 10:50:40.117879 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81424) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 10:50:40.118107 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81424) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 10:50:40.118200 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81423) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 10:50:40.118383 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81424) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 10:50:40.118463 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81423) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 10:50:40.118661 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81424) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 10:50:40.118741 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81423) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 10:50:40.119050 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81424) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 10:50:40.119333 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81424) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 10:50:40.119610 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81424) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 10:50:40.141310 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81423) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 10:50:40.142144 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81424) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 10:50:40.142821 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81423) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 26 10:50:40.143112 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81423) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 10:50:40.143608 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81424) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 26 10:50:40.143895 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81424) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 10:50:40.147831 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81423) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 26 10:50:40.148118 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81423) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 10:50:40.148405 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81423) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 10:50:40.148676 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81424) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 26 10:50:40.148962 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81424) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 10:50:40.149247 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81424) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 10:50:40.154844 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81423) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 26 10:50:40.155135 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81423) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 10:50:40.155674 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81424) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 26 10:50:40.155962 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81424) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 10:50:40.159253 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81423) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 26 10:50:40.159548 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81423) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 10:50:40.160386 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81423) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 10:50:40.160679 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81423) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 10:50:40.160767 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81424) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 26 10:50:40.161039 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81424) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 10:50:40.161121 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81423) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 10:50:40.161326 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81424) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 10:50:40.161417 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81423) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 10:50:40.161604 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81423) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 10:50:40.161684 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81424) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 10:50:40.161880 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81423) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 10:50:40.161968 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81424) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 10:50:40.162183 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81424) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 10:50:40.162457 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81424) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 10:50:40.162729 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81424) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 10:50:40.180615 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81424) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 10:50:40.180906 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81424) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 10:50:40.183816 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81423) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 10:50:40.184115 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81423) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 10:50:40.184831 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81424) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 26 10:50:40.188031 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81423) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 26 10:50:40.188125 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81424) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 10:50:40.188376 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81424) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 10:50:40.191329 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81423) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 10:50:40.191631 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81423) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 10:50:40.192618 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81424) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 26 10:50:40.195775 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81424) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 26 10:50:40.195867 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81423) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 26 10:50:40.198990 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81423) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 26 10:50:40.199078 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81424) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 10:50:40.199322 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81424) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 10:50:40.199609 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81424) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 10:50:40.199904 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81424) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 10:50:40.200202 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81424) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 10:50:40.200500 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81424) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 10:50:40.202259 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81423) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 10:50:40.202569 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81423) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 10:50:40.202861 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81423) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 10:50:40.203161 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81423) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 10:50:40.203453 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81423) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 10:50:40.203753 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81423) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 10:50:40.216849 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81424) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 26 10:50:40.217138 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81424) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 10:50:40.217424 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81424) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 10:50:40.217695 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81424) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 10:50:40.220099 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81423) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 26 10:50:40.220390 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81423) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 10:50:40.220674 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81423) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 10:50:40.220952 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81423) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 10:50:40.225366 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81424) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 10:50:40.226943 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81424) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 10:50:40.227223 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81424) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 10:50:40.228607 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81423) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 10:50:40.229865 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81424) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 26 10:50:40.230176 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81423) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 10:50:40.230263 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81424) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 10:50:40.230461 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81423) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 10:50:40.230558 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81424) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 10:50:40.230754 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81424) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 10:50:40.231048 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81424) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 10:50:40.233376 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81423) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 26 10:50:40.233666 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81423) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 10:50:40.233965 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81423) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 10:50:40.234279 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81423) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 10:50:40.234578 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81423) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 10:50:40.243000 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81424) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 26 10:50:40.243288 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81424) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 10:50:40.243572 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81424) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 10:50:40.243856 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81424) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 10:50:40.244142 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81424) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 10:50:40.244421 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81424) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 10:50:40.244710 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81424) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 10:50:40.245884 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81423) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 26 10:50:40.246199 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81423) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 10:50:40.246489 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81423) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 10:50:40.246774 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81423) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 10:50:40.247061 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81423) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 10:50:40.247351 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81423) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 10:50:40.247642 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81423) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 10:50:40.260582 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81424) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 26 10:50:40.263223 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81423) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 26 10:50:40.263627 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81424) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 26 10:50:40.266266 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81423) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 26 10:50:40.266537 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81424) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 10:50:40.267971 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81424) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 10:50:40.269155 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81423) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 10:50:40.269990 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81424) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 26 10:50:40.270276 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81424) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 10:50:40.270557 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81424) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 10:50:40.271354 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81423) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 10:50:40.272846 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81423) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 26 10:50:40.273135 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81423) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 10:50:40.273422 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81423) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 10:50:40.277020 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81424) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 10:50:40.278507 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81424) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 26 10:50:40.279918 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81423) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 10:50:40.281380 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81423) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 26 10:50:40.281462 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81424) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 10:50:40.281735 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81424) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 10:50:40.282056 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81424) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 10:50:40.282340 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81424) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 10:50:40.284324 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81423) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 10:50:40.284623 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81423) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 10:50:40.285606 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81423) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 10:50:40.286015 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81423) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 10:50:40.290913 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81424) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 26 10:50:40.291206 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81424) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 10:50:40.291491 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81424) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 10:50:40.293951 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81423) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 26 10:50:40.294279 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81423) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 10:50:40.294569 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81423) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 10:50:40.298394 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81424) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 26 10:50:40.298681 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81424) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 10:50:40.298965 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81424) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 10:50:40.299244 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81424) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 10:50:40.299527 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81424) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 10:50:40.299800 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81424) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 10:50:40.300075 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81424) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 10:50:40.300364 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81424) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 10:50:40.300678 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81424) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 10:50:40.300968 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81424) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 10:50:40.301242 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81424) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 10:50:40.301505 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81423) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 26 10:50:40.301587 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81424) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 10:50:40.301777 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81424) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 10:50:40.301894 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81423) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 10:50:40.302049 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81424) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 10:50:40.302129 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81423) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 10:50:40.302374 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81423) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 10:50:40.302654 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81423) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 10:50:40.302932 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81423) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 10:50:40.303212 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81423) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 10:50:40.303516 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81423) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 10:50:40.303798 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81423) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 10:50:40.304092 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81423) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 10:50:40.304369 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81423) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 10:50:40.304636 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81423) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 10:50:40.304908 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81423) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 10:50:40.305171 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81423) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 10:50:40.337213 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x55b4db9489c0 pid: 81424 (default app) Aug 26 10:50:40.339681 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x55b4db9489c0 pid: 81423 (default app) Aug 26 10:50:40.340616 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d389b250-2c2f-4b99-9beb-48b504008bc7 None None] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:50:40.340904 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d389b250-2c2f-4b99-9beb-48b504008bc7 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:50:40.341185 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d389b250-2c2f-4b99-9beb-48b504008bc7 None None] PATH_INFO: `/v3` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:50:40.345321 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1/1] 10.222.0.40 () {50 vars in 791 bytes} [Fri Aug 26 10:50:40 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 10:50:46.994946 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4f84e1-f3be-4e5c-b9d4-cdd3042bdb73 None None] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:50:46.995495 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4f84e1-f3be-4e5c-b9d4-cdd3042bdb73 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:50:46.995495 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4f84e1-f3be-4e5c-b9d4-cdd3042bdb73 None None] PATH_INFO: `/` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:50:46.999796 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1/2] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 26 10:50:46 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 26 10:50:47.005802 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38086d8c-4306-40e0-8af5-f5ea310b9f09 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:50:47.006218 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38086d8c-4306-40e0-8af5-f5ea310b9f09 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:50:47.006390 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38086d8c-4306-40e0-8af5-f5ea310b9f09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:50:47.076895 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: INFO dbcounter [None req-38086d8c-4306-40e0-8af5-f5ea310b9f09 None None] Registered counter for database keystone Aug 26 10:50:47.095573 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG oslo_db.sqlalchemy.engines [None req-38086d8c-4306-40e0-8af5-f5ea310b9f09 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81424) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 26 10:50:47.102102 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writer thread running {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 26 10:50:47.283367 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG passlib.handlers.bcrypt [None req-38086d8c-4306-40e0-8af5-f5ea310b9f09 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=81424) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Aug 26 10:50:47.283709 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG passlib.handlers.bcrypt [None req-38086d8c-4306-40e0-8af5-f5ea310b9f09 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81424) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Aug 26 10:50:47.338149 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-38086d8c-4306-40e0-8af5-f5ea310b9f09 None None] MFA Rules not processed for user `ca155a48e2024c32af3fd2e19e5fac11`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 10:50:47.339783 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-38086d8c-4306-40e0-8af5-f5ea310b9f09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 10:50:47.467277 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 2/3] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 26 10:50:47 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 462 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Aug 26 10:50:47.898407 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4f84e1-f3be-4e5c-b9d4-cdd3042bdb73 None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:50:47.966522 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: INFO dbcounter [None req-5b4f84e1-f3be-4e5c-b9d4-cdd3042bdb73 None None] Registered counter for database keystone Aug 26 10:50:47.988473 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG oslo_db.sqlalchemy.engines [None req-5b4f84e1-f3be-4e5c-b9d4-cdd3042bdb73 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81423) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 26 10:50:47.994624 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writer thread running {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 26 10:50:48.116716 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4f84e1-f3be-4e5c-b9d4-cdd3042bdb73 None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:50:48.121962 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c400f8-4694-46f2-b283-eeeb11d13b7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:50:48.122621 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c400f8-4694-46f2-b283-eeeb11d13b7a admin admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:50:48.122901 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c400f8-4694-46f2-b283-eeeb11d13b7a admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:50:48.123174 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c400f8-4694-46f2-b283-eeeb11d13b7a admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:50:48.131623 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c400f8-4694-46f2-b283-eeeb11d13b7a admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:50:48.183627 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: WARNING keystone.common.rbac_enforcer.enforcer [None req-51c400f8-4694-46f2-b283-eeeb11d13b7a admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Aug 26 10:50:48.183969 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c400f8-4694-46f2-b283-eeeb11d13b7a admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:50:48.193551 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 2/4] 10.222.0.40 () {58 vars in 1324 bytes} [Fri Aug 26 10:50:47 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 26 10:50:48.199570 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38086d8c-4306-40e0-8af5-f5ea310b9f09 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:50:48.214884 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38086d8c-4306-40e0-8af5-f5ea310b9f09 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:50:48.219523 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4fbd3cf-7d45-4c91-8548-e6c89beec6d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:50:48.220645 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4fbd3cf-7d45-4c91-8548-e6c89beec6d2 admin admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:50:48.220931 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4fbd3cf-7d45-4c91-8548-e6c89beec6d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:50:48.221206 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4fbd3cf-7d45-4c91-8548-e6c89beec6d2 admin admin] PATH_INFO: `/v3/projects/f3942b05c97e4a1ca048ce3a0379b7a7` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:50:48.223524 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4fbd3cf-7d45-4c91-8548-e6c89beec6d2 admin admin] RBAC: Authorizing `identity:get_project(project_id=f3942b05c97e4a1ca048ce3a0379b7a7)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:50:48.276625 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: WARNING keystone.common.rbac_enforcer.enforcer [None req-a4fbd3cf-7d45-4c91-8548-e6c89beec6d2 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Aug 26 10:50:48.276973 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4fbd3cf-7d45-4c91-8548-e6c89beec6d2 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:50:48.281488 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 3/5] 10.222.0.40 () {56 vars in 1207 bytes} [Fri Aug 26 10:50:48 2022] GET /identity/v3/projects/f3942b05c97e4a1ca048ce3a0379b7a7 => generated 344 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 10:50:50.161939 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28d91ab-4768-4793-aaf2-fc87a7935d84 None None] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:50:50.162470 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28d91ab-4768-4793-aaf2-fc87a7935d84 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:50:50.162470 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28d91ab-4768-4793-aaf2-fc87a7935d84 None None] PATH_INFO: `/` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:50:50.166525 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 3/6] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 26 10:50:50 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 26 10:50:50.171953 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d962dc81-7dd2-4e44-9871-8f18ff414fe2 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:50:50.172379 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d962dc81-7dd2-4e44-9871-8f18ff414fe2 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:50:50.172503 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d962dc81-7dd2-4e44-9871-8f18ff414fe2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:50:50.192281 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-d962dc81-7dd2-4e44-9871-8f18ff414fe2 None None] MFA Rules not processed for user `ca155a48e2024c32af3fd2e19e5fac11`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 10:50:50.193802 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-d962dc81-7dd2-4e44-9871-8f18ff414fe2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 10:50:50.201652 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 4/7] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 26 10:50:50 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Aug 26 10:50:50.477008 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28d91ab-4768-4793-aaf2-fc87a7935d84 None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:50:50.490335 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28d91ab-4768-4793-aaf2-fc87a7935d84 None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:50:50.494553 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318c77d2-8c95-4768-805e-f2262278de73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:50:50.495148 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318c77d2-8c95-4768-805e-f2262278de73 admin admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:50:50.495426 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318c77d2-8c95-4768-805e-f2262278de73 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:50:50.495699 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318c77d2-8c95-4768-805e-f2262278de73 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:50:50.503341 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-318c77d2-8c95-4768-805e-f2262278de73 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:50:50.505226 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-318c77d2-8c95-4768-805e-f2262278de73 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:50:50.514280 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 4/8] 10.222.0.40 () {58 vars in 1324 bytes} [Fri Aug 26 10:50:50 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 26 10:50:50.519571 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d962dc81-7dd2-4e44-9871-8f18ff414fe2 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:50:50.528621 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d962dc81-7dd2-4e44-9871-8f18ff414fe2 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:50:50.532956 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26762d67-e4c3-433e-a43c-916567a6e8af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:50:50.533574 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26762d67-e4c3-433e-a43c-916567a6e8af admin admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:50:50.533850 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26762d67-e4c3-433e-a43c-916567a6e8af admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:50:50.534155 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26762d67-e4c3-433e-a43c-916567a6e8af admin admin] PATH_INFO: `/v3/users/ca155a48e2024c32af3fd2e19e5fac11` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:50:50.536081 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26762d67-e4c3-433e-a43c-916567a6e8af admin admin] RBAC: Authorizing `identity:get_user(user_id=ca155a48e2024c32af3fd2e19e5fac11)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:50:50.537898 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26762d67-e4c3-433e-a43c-916567a6e8af admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:50:50.539451 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 5/9] 10.222.0.40 () {56 vars in 1198 bytes} [Fri Aug 26 10:50:50 2022] GET /identity/v3/users/ca155a48e2024c32af3fd2e19e5fac11 => generated 247 bytes in 20 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 10:50:52.502602 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5691e1-e23f-4f17-8b75-366a6910568b None None] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:50:52.503130 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5691e1-e23f-4f17-8b75-366a6910568b None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:50:52.503130 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5691e1-e23f-4f17-8b75-366a6910568b None None] PATH_INFO: `/` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:50:52.505182 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 5/10] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 26 10:50:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 26 10:50:52.510373 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96a6da0-0b58-43eb-bd1a-06c6afb5e397 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:50:52.510778 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96a6da0-0b58-43eb-bd1a-06c6afb5e397 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:50:52.510929 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96a6da0-0b58-43eb-bd1a-06c6afb5e397 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:50:52.523872 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfcc3f8e-9fb5-4ec8-b502-0f9dab25d17f None None] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:50:52.524153 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfcc3f8e-9fb5-4ec8-b502-0f9dab25d17f None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:50:52.524429 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfcc3f8e-9fb5-4ec8-b502-0f9dab25d17f None None] PATH_INFO: `/` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:50:52.525748 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 6/11] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 26 10:50:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 10:50:52.528947 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-d96a6da0-0b58-43eb-bd1a-06c6afb5e397 None None] MFA Rules not processed for user `ca155a48e2024c32af3fd2e19e5fac11`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 10:50:52.530182 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-d96a6da0-0b58-43eb-bd1a-06c6afb5e397 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 10:50:52.530812 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82bb4d9d-5acc-4521-91f2-44ac62965411 None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:50:52.531097 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82bb4d9d-5acc-4521-91f2-44ac62965411 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:50:52.531375 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82bb4d9d-5acc-4521-91f2-44ac62965411 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:50:52.573769 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG passlib.handlers.bcrypt [None req-82bb4d9d-5acc-4521-91f2-44ac62965411 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=81423) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Aug 26 10:50:52.574230 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG passlib.handlers.bcrypt [None req-82bb4d9d-5acc-4521-91f2-44ac62965411 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81423) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Aug 26 10:50:52.610842 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 6/12] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 26 10:50:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 10:50:52.620396 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-82bb4d9d-5acc-4521-91f2-44ac62965411 None None] MFA Rules not processed for user `ca155a48e2024c32af3fd2e19e5fac11`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 10:50:52.622037 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-82bb4d9d-5acc-4521-91f2-44ac62965411 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 10:50:52.721198 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 7/13] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 26 10:50:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 26 10:50:52.875989 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96a6da0-0b58-43eb-bd1a-06c6afb5e397 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:50:52.899542 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:50:52.980259 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96a6da0-0b58-43eb-bd1a-06c6afb5e397 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:50:52.993398 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82bb4d9d-5acc-4521-91f2-44ac62965411 None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:50:53.012832 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81255d9a-3e3b-4b36-87ec-ad4865d33fc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:50:53.013462 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81255d9a-3e3b-4b36-87ec-ad4865d33fc2 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:50:53.013738 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81255d9a-3e3b-4b36-87ec-ad4865d33fc2 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:50:53.014048 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81255d9a-3e3b-4b36-87ec-ad4865d33fc2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:50:53.026343 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81255d9a-3e3b-4b36-87ec-ad4865d33fc2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 10:50:53.036109 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application [None req-81255d9a-3e3b-4b36-87ec-ad4865d33fc2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 26 10:50:53.036109 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 10:50:53.036109 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 10:50:53.036109 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 10:50:53.036109 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 10:50:53.036109 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 10:50:53.036109 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 10:50:53.036109 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 10:50:53.036109 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 10:50:53.036109 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 10:50:53.036109 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 10:50:53.036109 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 10:50:53.036109 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 10:50:53.036109 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 10:50:53.036109 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 10:50:53.036109 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 10:50:53.037054 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 10:50:53.037054 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 10:50:53.037054 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 10:50:53.037054 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 10:50:53.037054 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 10:50:53.037054 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 10:50:53.037054 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 10:50:53.037054 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 10:50:53.037054 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 10:50:53.037054 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 10:50:53.037054 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 10:50:53.037054 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application with Lock( Aug 26 10:50:53.037054 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 10:50:53.037054 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._enter() Aug 26 10:50:53.037054 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 10:50:53.037054 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 10:50:53.038119 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 10:50:53.038119 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.creator() Aug 26 10:50:53.038119 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 10:50:53.038119 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application created_value = creator( Aug 26 10:50:53.038119 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 10:50:53.038119 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 10:50:53.038119 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 10:50:53.038119 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 10:50:53.038119 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 10:50:53.038119 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 10:50:53.038119 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 26 10:50:53.038119 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Aug 26 10:50:53.038119 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 7/14] 10.222.0.40 () {56 vars in 1096 bytes} [Fri Aug 26 10:50:52 2022] GET /identity/v3/roles/admin => generated 83 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 10:50:53.043036 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81255d9a-3e3b-4b36-87ec-ad4865d33fc2 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:50:53.099324 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82bb4d9d-5acc-4521-91f2-44ac62965411 None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:50:53.101615 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:50:53.127311 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:50:53.139621 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22566c65-7e51-4fa0-baa6-10433732ebbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:50:53.140480 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22566c65-7e51-4fa0-baa6-10433732ebbe None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:50:53.140771 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22566c65-7e51-4fa0-baa6-10433732ebbe None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:50:53.141045 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22566c65-7e51-4fa0-baa6-10433732ebbe None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:50:53.146782 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81255d9a-3e3b-4b36-87ec-ad4865d33fc2 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:50:53.180067 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f662fddb-5a95-462d-8566-7497c218083a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:50:53.180668 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f662fddb-5a95-462d-8566-7497c218083a None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:50:53.180941 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f662fddb-5a95-462d-8566-7497c218083a None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:50:53.181212 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f662fddb-5a95-462d-8566-7497c218083a None admin] PATH_INFO: `/v3/roles` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:50:53.181867 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f662fddb-5a95-462d-8566-7497c218083a None admin] RBAC: Adding query filter params (name=admin) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 10:50:53.182193 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f662fddb-5a95-462d-8566-7497c218083a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:50:53.183574 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f662fddb-5a95-462d-8566-7497c218083a None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:50:53.195450 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 8/15] 10.222.0.40 () {56 vars in 1099 bytes} [Fri Aug 26 10:50:53 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 10:50:53.200977 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f662fddb-5a95-462d-8566-7497c218083a None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:50:53.208759 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22566c65-7e51-4fa0-baa6-10433732ebbe None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:50:53.210311 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22566c65-7e51-4fa0-baa6-10433732ebbe None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:50:53.298917 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f662fddb-5a95-462d-8566-7497c218083a None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:50:53.309473 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 8/16] 10.222.0.40 () {58 vars in 1282 bytes} [Fri Aug 26 10:50:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 26 10:50:53.314828 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22566c65-7e51-4fa0-baa6-10433732ebbe None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:50:53.318260 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:50:53.335823 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1368e094-40bb-4783-81ee-cff636f5a96c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:50:53.336465 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1368e094-40bb-4783-81ee-cff636f5a96c None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:50:53.336744 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1368e094-40bb-4783-81ee-cff636f5a96c None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:50:53.337018 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1368e094-40bb-4783-81ee-cff636f5a96c None admin] PATH_INFO: `/v3/users/ca155a48e2024c32af3fd2e19e5fac11` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:50:53.338192 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1368e094-40bb-4783-81ee-cff636f5a96c None admin] RBAC: Authorizing `identity:get_user(user_id=ca155a48e2024c32af3fd2e19e5fac11)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:50:53.339645 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1368e094-40bb-4783-81ee-cff636f5a96c None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:50:53.341196 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 9/17] 10.222.0.40 () {56 vars in 1177 bytes} [Fri Aug 26 10:50:53 2022] GET /identity/v3/users/ca155a48e2024c32af3fd2e19e5fac11 => generated 247 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 10:50:53.346608 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1368e094-40bb-4783-81ee-cff636f5a96c None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:50:53.361425 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:50:53.415743 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22566c65-7e51-4fa0-baa6-10433732ebbe None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:50:53.447383 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1368e094-40bb-4783-81ee-cff636f5a96c None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:50:53.449883 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b641039-686f-48ee-8bdc-0c1a5dd561ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:50:53.450839 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b641039-686f-48ee-8bdc-0c1a5dd561ac None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:50:53.451124 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b641039-686f-48ee-8bdc-0c1a5dd561ac None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:50:53.451394 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b641039-686f-48ee-8bdc-0c1a5dd561ac None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:50:53.470180 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b641039-686f-48ee-8bdc-0c1a5dd561ac None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:50:53.471642 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b641039-686f-48ee-8bdc-0c1a5dd561ac None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:50:53.481365 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f35570-d86d-454b-b8f4-08d7846b48fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:50:53.482002 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f35570-d86d-454b-b8f4-08d7846b48fc None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:50:53.482287 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f35570-d86d-454b-b8f4-08d7846b48fc None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:50:53.482821 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f35570-d86d-454b-b8f4-08d7846b48fc None admin] PATH_INFO: `/v3/domains/default` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:50:53.483927 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f35570-d86d-454b-b8f4-08d7846b48fc None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:50:53.485341 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f35570-d86d-454b-b8f4-08d7846b48fc None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:50:53.486915 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 10/18] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Aug 26 10:50:53 2022] GET /identity/v3/domains/default => generated 199 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 10:50:53.489966 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application [None req-5b641039-686f-48ee-8bdc-0c1a5dd561ac None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 10:50:53.489966 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 10:50:53.489966 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 10:50:53.489966 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 10:50:53.489966 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 10:50:53.489966 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 10:50:53.489966 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 10:50:53.489966 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 10:50:53.489966 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 10:50:53.489966 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 10:50:53.489966 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 10:50:53.489966 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 10:50:53.489966 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 10:50:53.489966 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 10:50:53.489966 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 10:50:53.489966 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 10:50:53.490869 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 10:50:53.490869 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 10:50:53.490869 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 10:50:53.490869 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 10:50:53.490869 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 10:50:53.490869 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 10:50:53.490869 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 10:50:53.490869 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application with Lock( Aug 26 10:50:53.490869 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 10:50:53.490869 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._enter() Aug 26 10:50:53.490869 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 10:50:53.490869 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 10:50:53.490869 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 10:50:53.490869 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.creator() Aug 26 10:50:53.490869 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 10:50:53.490869 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application created_value = creator( Aug 26 10:50:53.490869 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 10:50:53.491847 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 10:50:53.491847 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 10:50:53.491847 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Aug 26 10:50:53.491847 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 9/19] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Aug 26 10:50:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 26 10:50:53.492758 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f35570-d86d-454b-b8f4-08d7846b48fc None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:50:53.496611 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b641039-686f-48ee-8bdc-0c1a5dd561ac None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:50:53.555542 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:50:53.574903 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:50:53.595447 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f35570-d86d-454b-b8f4-08d7846b48fc None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:50:53.600049 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b641039-686f-48ee-8bdc-0c1a5dd561ac None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:50:53.630473 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32eb534a-3636-4cd8-a3b2-3b5caea35fb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:50:53.631182 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32eb534a-3636-4cd8-a3b2-3b5caea35fb9 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:50:53.631462 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32eb534a-3636-4cd8-a3b2-3b5caea35fb9 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:50:53.631737 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32eb534a-3636-4cd8-a3b2-3b5caea35fb9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:50:53.632464 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32eb534a-3636-4cd8-a3b2-3b5caea35fb9 None admin] RBAC: Adding query filter params (role.id=1f97e4602e2e4d6498913a81f9ae8d6a, scope.domain.id=default, user.id=ca155a48e2024c32af3fd2e19e5fac11) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 10:50:53.632775 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32eb534a-3636-4cd8-a3b2-3b5caea35fb9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:50:53.633827 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a40fa25-68fc-4b80-8a05-5ee12b61f5a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:50:53.634248 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32eb534a-3636-4cd8-a3b2-3b5caea35fb9 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:50:53.634473 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a40fa25-68fc-4b80-8a05-5ee12b61f5a2 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:50:53.634754 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a40fa25-68fc-4b80-8a05-5ee12b61f5a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:50:53.635026 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a40fa25-68fc-4b80-8a05-5ee12b61f5a2 None admin] PATH_INFO: `/v3/domains` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:50:53.635667 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a40fa25-68fc-4b80-8a05-5ee12b61f5a2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 10:50:53.635969 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a40fa25-68fc-4b80-8a05-5ee12b61f5a2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:50:53.637332 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a40fa25-68fc-4b80-8a05-5ee12b61f5a2 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:50:53.640925 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 11/20] 10.222.0.40 () {56 vars in 1322 bytes} [Fri Aug 26 10:50:53 2022] GET /identity/v3/role_assignments?user.id=ca155a48e2024c32af3fd2e19e5fac11&scope.domain.id=default&role.id=1f97e4602e2e4d6498913a81f9ae8d6a => generated 233 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 10:50:53.656063 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 10/21] 10.222.0.40 () {56 vars in 1109 bytes} [Fri Aug 26 10:50:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 10:50:55.802697 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce77995-65eb-4154-bebd-329cda0f23db None None] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:50:55.803204 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce77995-65eb-4154-bebd-329cda0f23db None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:50:55.803305 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce77995-65eb-4154-bebd-329cda0f23db None None] PATH_INFO: `/` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:50:55.804832 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 12/22] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 26 10:50:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 10:50:55.810382 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2000c120-6bd0-4fb8-befb-51b480a81bad None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:50:55.810776 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2000c120-6bd0-4fb8-befb-51b480a81bad None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:50:55.810952 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2000c120-6bd0-4fb8-befb-51b480a81bad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:50:55.839236 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-2000c120-6bd0-4fb8-befb-51b480a81bad None None] MFA Rules not processed for user `ca155a48e2024c32af3fd2e19e5fac11`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 10:50:55.840633 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-2000c120-6bd0-4fb8-befb-51b480a81bad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 10:50:55.940722 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 11/23] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 26 10:50:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 26 10:50:56.047318 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a4d993-829a-49f1-8192-ac16dee6bc05 None None] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:50:56.047634 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a4d993-829a-49f1-8192-ac16dee6bc05 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:50:56.047882 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a4d993-829a-49f1-8192-ac16dee6bc05 None None] PATH_INFO: `/` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:50:56.049482 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 13/24] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 26 10:50:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 10:50:56.055584 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd04262c-903d-4f2e-a5f3-f0e25b5a7fd7 None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:50:56.056082 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd04262c-903d-4f2e-a5f3-f0e25b5a7fd7 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:50:56.056535 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd04262c-903d-4f2e-a5f3-f0e25b5a7fd7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:50:56.083024 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-cd04262c-903d-4f2e-a5f3-f0e25b5a7fd7 None None] MFA Rules not processed for user `ca155a48e2024c32af3fd2e19e5fac11`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 10:50:56.084251 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-cd04262c-903d-4f2e-a5f3-f0e25b5a7fd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 10:50:56.143967 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:50:56.178592 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 12/25] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 26 10:50:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 10:50:56.241575 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a52ed6-3615-46df-975a-5799631c0e87 None None] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:50:56.242267 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a52ed6-3615-46df-975a-5799631c0e87 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:50:56.242755 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a52ed6-3615-46df-975a-5799631c0e87 None None] PATH_INFO: `/` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:50:56.244299 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 14/26] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 26 10:50:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 10:50:56.249813 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fecdd03-b2ff-4f70-b934-9e86453e1245 None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:50:56.250293 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fecdd03-b2ff-4f70-b934-9e86453e1245 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:50:56.250734 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fecdd03-b2ff-4f70-b934-9e86453e1245 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:50:56.279340 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-1fecdd03-b2ff-4f70-b934-9e86453e1245 None None] MFA Rules not processed for user `ca155a48e2024c32af3fd2e19e5fac11`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 10:50:56.281706 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-1fecdd03-b2ff-4f70-b934-9e86453e1245 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 10:50:56.368446 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 13/27] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 26 10:50:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 10:50:56.384930 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a52ed6-3615-46df-975a-5799631c0e87 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:50:56.511645 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a52ed6-3615-46df-975a-5799631c0e87 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:50:56.540659 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9ba9d8-4bce-4d3e-a73d-2fe03038e973 None None] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:50:56.541185 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9ba9d8-4bce-4d3e-a73d-2fe03038e973 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:50:56.541603 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9ba9d8-4bce-4d3e-a73d-2fe03038e973 None None] PATH_INFO: `/` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:50:56.543139 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 14/28] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 26 10:50:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 26 10:50:56.556633 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746abf81-d9df-4f7e-9f73-6404f98f1923 None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:50:56.557154 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746abf81-d9df-4f7e-9f73-6404f98f1923 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:50:56.557588 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746abf81-d9df-4f7e-9f73-6404f98f1923 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:50:56.583945 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f0d252-3725-4e5c-a77d-cb09638daa13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:50:56.584849 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f0d252-3725-4e5c-a77d-cb09638daa13 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:50:56.585275 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f0d252-3725-4e5c-a77d-cb09638daa13 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:50:56.585718 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f0d252-3725-4e5c-a77d-cb09638daa13 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:50:56.587219 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-746abf81-d9df-4f7e-9f73-6404f98f1923 None None] MFA Rules not processed for user `ca155a48e2024c32af3fd2e19e5fac11`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 10:50:56.592013 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-746abf81-d9df-4f7e-9f73-6404f98f1923 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 10:50:56.605346 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f0d252-3725-4e5c-a77d-cb09638daa13 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 10:50:56.610770 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application [None req-78f0d252-3725-4e5c-a77d-cb09638daa13 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 26 10:50:56.610770 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 10:50:56.610770 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 10:50:56.610770 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 10:50:56.610770 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 10:50:56.610770 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 10:50:56.610770 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 10:50:56.610770 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 10:50:56.610770 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 10:50:56.610770 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 10:50:56.610770 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 10:50:56.610770 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 10:50:56.610770 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 10:50:56.610770 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 10:50:56.610770 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 10:50:56.610770 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 10:50:56.611827 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 10:50:56.611827 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 10:50:56.611827 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 10:50:56.611827 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 10:50:56.611827 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 10:50:56.611827 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 10:50:56.611827 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 10:50:56.611827 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 10:50:56.611827 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 10:50:56.611827 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 10:50:56.611827 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 10:50:56.611827 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application with Lock( Aug 26 10:50:56.611827 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 10:50:56.611827 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._enter() Aug 26 10:50:56.611827 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 10:50:56.611827 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 10:50:56.613646 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 10:50:56.613646 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.creator() Aug 26 10:50:56.613646 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 10:50:56.613646 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application created_value = creator( Aug 26 10:50:56.613646 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 10:50:56.613646 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 10:50:56.613646 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 10:50:56.613646 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 10:50:56.613646 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 10:50:56.613646 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 10:50:56.613646 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 26 10:50:56.613646 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Aug 26 10:50:56.622238 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 15/29] 10.222.0.40 () {56 vars in 1096 bytes} [Fri Aug 26 10:50:56 2022] GET /identity/v3/roles/admin => generated 83 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 10:50:56.628024 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f0d252-3725-4e5c-a77d-cb09638daa13 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:50:56.643139 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:50:56.727265 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 15/30] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 26 10:50:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 26 10:50:56.731005 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746abf81-d9df-4f7e-9f73-6404f98f1923 None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:50:56.797417 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:50:56.805903 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f0d252-3725-4e5c-a77d-cb09638daa13 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:50:56.875079 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76974e76-baa9-4510-9d70-87463accbe1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:50:56.875614 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76974e76-baa9-4510-9d70-87463accbe1f None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:50:56.875891 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76974e76-baa9-4510-9d70-87463accbe1f None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:50:56.876163 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76974e76-baa9-4510-9d70-87463accbe1f None admin] PATH_INFO: `/v3/roles` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:50:56.876832 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76974e76-baa9-4510-9d70-87463accbe1f None admin] RBAC: Adding query filter params (name=admin) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 10:50:56.877131 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76974e76-baa9-4510-9d70-87463accbe1f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:50:56.879271 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76974e76-baa9-4510-9d70-87463accbe1f None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:50:56.893420 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 16/31] 10.222.0.40 () {56 vars in 1099 bytes} [Fri Aug 26 10:50:56 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 10:50:56.896164 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76974e76-baa9-4510-9d70-87463accbe1f None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:50:56.932163 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746abf81-d9df-4f7e-9f73-6404f98f1923 None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:50:57.006484 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72297d8c-440b-408f-b45e-575d10d38da7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:50:57.007383 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72297d8c-440b-408f-b45e-575d10d38da7 None admin] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:50:57.008489 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72297d8c-440b-408f-b45e-575d10d38da7 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:50:57.008910 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72297d8c-440b-408f-b45e-575d10d38da7 None admin] PATH_INFO: `/v3/roles` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:50:57.010387 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72297d8c-440b-408f-b45e-575d10d38da7 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:50:57.012082 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72297d8c-440b-408f-b45e-575d10d38da7 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:50:57.024611 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:50:57.070193 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76974e76-baa9-4510-9d70-87463accbe1f None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:50:57.111705 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c552b4-f738-4380-9e11-df7b2cce5d0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:50:57.112656 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c552b4-f738-4380-9e11-df7b2cce5d0d None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:50:57.113657 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c552b4-f738-4380-9e11-df7b2cce5d0d None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:50:57.114118 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c552b4-f738-4380-9e11-df7b2cce5d0d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:50:57.125276 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c552b4-f738-4380-9e11-df7b2cce5d0d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:50:57.127023 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c552b4-f738-4380-9e11-df7b2cce5d0d None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:50:57.145990 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application [None req-a1c552b4-f738-4380-9e11-df7b2cce5d0d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 10:50:57.145990 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 10:50:57.145990 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 10:50:57.145990 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 10:50:57.145990 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 10:50:57.145990 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 10:50:57.145990 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 10:50:57.145990 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 10:50:57.145990 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 10:50:57.145990 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 10:50:57.145990 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 10:50:57.145990 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 10:50:57.145990 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 10:50:57.145990 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 10:50:57.145990 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 10:50:57.145990 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 10:50:57.147098 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 10:50:57.147098 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 10:50:57.147098 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 10:50:57.147098 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 10:50:57.147098 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 10:50:57.147098 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 10:50:57.147098 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 10:50:57.147098 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application with Lock( Aug 26 10:50:57.147098 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 10:50:57.147098 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._enter() Aug 26 10:50:57.147098 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 10:50:57.147098 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 10:50:57.147098 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 10:50:57.147098 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.creator() Aug 26 10:50:57.147098 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 10:50:57.147098 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application created_value = creator( Aug 26 10:50:57.147098 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 10:50:57.148075 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 10:50:57.148075 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 10:50:57.148075 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Aug 26 10:50:57.149819 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 17/32] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Aug 26 10:50:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 10:50:57.150752 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c552b4-f738-4380-9e11-df7b2cce5d0d None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:50:57.165639 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 16/33] 10.222.0.40 () {60 vars in 1131 bytes} [Fri Aug 26 10:50:56 2022] POST /identity/v3/roles => generated 219 bytes in 435 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 10:50:57.166469 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72297d8c-440b-408f-b45e-575d10d38da7 None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:50:57.282959 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:50:57.290025 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72297d8c-440b-408f-b45e-575d10d38da7 None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:50:57.328551 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d061cde-eb12-4c7e-98cb-a2c7dce02010 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:50:57.329176 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d061cde-eb12-4c7e-98cb-a2c7dce02010 None admin] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:50:57.329476 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d061cde-eb12-4c7e-98cb-a2c7dce02010 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:50:57.329737 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d061cde-eb12-4c7e-98cb-a2c7dce02010 None admin] PATH_INFO: `/v3/roles` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:50:57.330454 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d061cde-eb12-4c7e-98cb-a2c7dce02010 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:50:57.332631 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d061cde-eb12-4c7e-98cb-a2c7dce02010 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:50:57.365034 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 17/34] 10.222.0.40 () {60 vars in 1131 bytes} [Fri Aug 26 10:50:57 2022] POST /identity/v3/roles => generated 225 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 10:50:57.368577 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d061cde-eb12-4c7e-98cb-a2c7dce02010 None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:50:57.370911 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c552b4-f738-4380-9e11-df7b2cce5d0d None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:50:57.432151 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:50:57.498784 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3aa2cdb-f574-4c9a-b049-a2271eb3cc3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:50:57.499458 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3aa2cdb-f574-4c9a-b049-a2271eb3cc3e None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:50:57.499745 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3aa2cdb-f574-4c9a-b049-a2271eb3cc3e None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:50:57.500026 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3aa2cdb-f574-4c9a-b049-a2271eb3cc3e None admin] PATH_INFO: `/v3/users/ca155a48e2024c32af3fd2e19e5fac11` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:50:57.501342 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3aa2cdb-f574-4c9a-b049-a2271eb3cc3e None admin] RBAC: Authorizing `identity:get_user(user_id=ca155a48e2024c32af3fd2e19e5fac11)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:50:57.502876 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3aa2cdb-f574-4c9a-b049-a2271eb3cc3e None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:50:57.505440 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 18/35] 10.222.0.40 () {56 vars in 1177 bytes} [Fri Aug 26 10:50:57 2022] GET /identity/v3/users/ca155a48e2024c32af3fd2e19e5fac11 => generated 247 bytes in 355 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 10:50:57.515071 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3aa2cdb-f574-4c9a-b049-a2271eb3cc3e None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:50:57.517627 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d061cde-eb12-4c7e-98cb-a2c7dce02010 None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:50:57.570022 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b778781b-c6ad-4b25-bbde-3065f2021b5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:50:57.570766 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b778781b-c6ad-4b25-bbde-3065f2021b5f None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:50:57.571055 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b778781b-c6ad-4b25-bbde-3065f2021b5f None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:50:57.571328 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b778781b-c6ad-4b25-bbde-3065f2021b5f None admin] PATH_INFO: `/v3/domains` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:50:57.571991 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b778781b-c6ad-4b25-bbde-3065f2021b5f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 10:50:57.572296 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b778781b-c6ad-4b25-bbde-3065f2021b5f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:50:57.573699 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b778781b-c6ad-4b25-bbde-3065f2021b5f None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:50:57.591192 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 18/36] 10.222.0.40 () {56 vars in 1109 bytes} [Fri Aug 26 10:50:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 10:50:57.594550 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989ab663-d0b2-4651-848a-2d648694334e None None] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:50:57.594928 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989ab663-d0b2-4651-848a-2d648694334e None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:50:57.595215 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989ab663-d0b2-4651-848a-2d648694334e None None] PATH_INFO: `/` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:50:57.596666 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 19/37] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 26 10:50:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 26 10:50:57.599522 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989ab663-d0b2-4651-848a-2d648694334e None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:50:57.629886 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:50:57.684255 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3aa2cdb-f574-4c9a-b049-a2271eb3cc3e None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:50:57.719039 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a39745-b9b5-4ea5-999c-1e754f0b613d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:50:57.719680 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a39745-b9b5-4ea5-999c-1e754f0b613d None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:50:57.719960 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a39745-b9b5-4ea5-999c-1e754f0b613d None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:50:57.720234 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a39745-b9b5-4ea5-999c-1e754f0b613d None admin] PATH_INFO: `/v3/domains/default` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:50:57.721360 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a39745-b9b5-4ea5-999c-1e754f0b613d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:50:57.722956 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a39745-b9b5-4ea5-999c-1e754f0b613d None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:50:57.724613 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 19/38] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Aug 26 10:50:57 2022] GET /identity/v3/domains/default => generated 199 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 10:50:57.725767 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e50dbb-6e4b-4cc8-b3c6-f2eae32adf55 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:50:57.726065 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e50dbb-6e4b-4cc8-b3c6-f2eae32adf55 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:50:57.726339 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e50dbb-6e4b-4cc8-b3c6-f2eae32adf55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:50:57.735585 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989ab663-d0b2-4651-848a-2d648694334e None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:50:57.745235 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-79e50dbb-6e4b-4cc8-b3c6-f2eae32adf55 None None] MFA Rules not processed for user `ca155a48e2024c32af3fd2e19e5fac11`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 10:50:57.746485 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-79e50dbb-6e4b-4cc8-b3c6-f2eae32adf55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 10:50:57.776125 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3573e90-796d-4dc2-a41c-9036aab14870 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:50:57.776741 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3573e90-796d-4dc2-a41c-9036aab14870 None admin] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:50:57.777021 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3573e90-796d-4dc2-a41c-9036aab14870 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:50:57.777294 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3573e90-796d-4dc2-a41c-9036aab14870 None admin] PATH_INFO: `/v3/projects` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:50:57.778225 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3573e90-796d-4dc2-a41c-9036aab14870 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:50:57.779675 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3573e90-796d-4dc2-a41c-9036aab14870 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:50:57.796425 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:50:57.819696 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 20/39] 10.222.0.40 () {60 vars in 1141 bytes} [Fri Aug 26 10:50:57 2022] POST /identity/v3/projects => generated 301 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 26 10:50:57.819696 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57daad0e-c624-43cc-ba09-8764d8f745e3 None None] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:50:57.819696 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57daad0e-c624-43cc-ba09-8764d8f745e3 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:50:57.819696 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57daad0e-c624-43cc-ba09-8764d8f745e3 None None] PATH_INFO: `/` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:50:57.819696 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 21/40] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 26 10:50:57 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 10:50:57.819696 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57daad0e-c624-43cc-ba09-8764d8f745e3 None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:50:57.830165 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 20/41] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 26 10:50:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 10:50:57.830165 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd8b533-48e4-47e5-a1a5-3bee6ba89957 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:50:57.830165 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd8b533-48e4-47e5-a1a5-3bee6ba89957 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:50:57.830165 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd8b533-48e4-47e5-a1a5-3bee6ba89957 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:50:57.848483 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-8dd8b533-48e4-47e5-a1a5-3bee6ba89957 None None] MFA Rules not processed for user `ca155a48e2024c32af3fd2e19e5fac11`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 10:50:57.849791 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-8dd8b533-48e4-47e5-a1a5-3bee6ba89957 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 10:50:57.902136 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:50:57.930740 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 21/42] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 26 10:50:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 10:50:57.947591 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57daad0e-c624-43cc-ba09-8764d8f745e3 None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:50:57.986461 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1156b6-0d80-4653-a2ae-8b4331a6c5aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:50:57.987378 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1156b6-0d80-4653-a2ae-8b4331a6c5aa None admin] REQUEST_METHOD: `PUT` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:50:57.988492 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1156b6-0d80-4653-a2ae-8b4331a6c5aa None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:50:57.988914 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1156b6-0d80-4653-a2ae-8b4331a6c5aa None admin] PATH_INFO: `/v3/domains/default/users/ca155a48e2024c32af3fd2e19e5fac11/roles/1f97e4602e2e4d6498913a81f9ae8d6a` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:50:57.991283 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e1156b6-0d80-4653-a2ae-8b4331a6c5aa None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=ca155a48e2024c32af3fd2e19e5fac11, role_id=1f97e4602e2e4d6498913a81f9ae8d6a)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:50:57.992904 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e1156b6-0d80-4653-a2ae-8b4331a6c5aa None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:50:58.013396 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 22/43] 10.222.0.40 () {58 vars in 1361 bytes} [Fri Aug 26 10:50:57 2022] PUT /identity/v3/domains/default/users/ca155a48e2024c32af3fd2e19e5fac11/roles/1f97e4602e2e4d6498913a81f9ae8d6a => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 10:50:58.109325 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd8b533-48e4-47e5-a1a5-3bee6ba89957 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:50:58.201944 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1156b6-0d80-4653-a2ae-8b4331a6c5aa None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:50:58.218406 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd8b533-48e4-47e5-a1a5-3bee6ba89957 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:50:58.235675 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:50:58.257427 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed845b8-9120-4723-82fc-b94a9814a978 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:50:58.258092 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed845b8-9120-4723-82fc-b94a9814a978 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:50:58.258373 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed845b8-9120-4723-82fc-b94a9814a978 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:50:58.258643 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed845b8-9120-4723-82fc-b94a9814a978 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:50:58.259753 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed845b8-9120-4723-82fc-b94a9814a978 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:50:58.261196 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed845b8-9120-4723-82fc-b94a9814a978 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:50:58.263045 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 22/44] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Aug 26 10:50:58 2022] GET /identity/v3/domains/default => generated 199 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 10:50:58.268359 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed845b8-9120-4723-82fc-b94a9814a978 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:50:58.312790 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1156b6-0d80-4653-a2ae-8b4331a6c5aa None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:50:58.341914 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:50:58.353743 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f027ac-866d-4cdc-a270-cc8bfd9005a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:50:58.354398 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f027ac-866d-4cdc-a270-cc8bfd9005a7 None admin] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:50:58.354679 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f027ac-866d-4cdc-a270-cc8bfd9005a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:50:58.354956 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f027ac-866d-4cdc-a270-cc8bfd9005a7 None admin] PATH_INFO: `/v3/roles` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:50:58.355640 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f027ac-866d-4cdc-a270-cc8bfd9005a7 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:50:58.357039 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f027ac-866d-4cdc-a270-cc8bfd9005a7 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:50:58.373401 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 23/45] 10.222.0.40 () {60 vars in 1131 bytes} [Fri Aug 26 10:50:58 2022] POST /identity/v3/roles => generated 223 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 26 10:50:58.379564 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed845b8-9120-4723-82fc-b94a9814a978 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:50:58.419270 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1192627a-0b2e-4521-b179-5de01858e310 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:50:58.419956 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1192627a-0b2e-4521-b179-5de01858e310 None admin] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:50:58.420243 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1192627a-0b2e-4521-b179-5de01858e310 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:50:58.420515 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1192627a-0b2e-4521-b179-5de01858e310 None admin] PATH_INFO: `/v3/projects` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:50:58.421189 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1192627a-0b2e-4521-b179-5de01858e310 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:50:58.422717 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1192627a-0b2e-4521-b179-5de01858e310 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:50:58.668581 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 23/46] 10.222.0.40 () {60 vars in 1141 bytes} [Fri Aug 26 10:50:58 2022] POST /identity/v3/projects => generated 312 bytes in 401 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 26 10:50:59.952165 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-badf4596-4243-48bc-8058-0f02f6d61b2f None None] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:50:59.952656 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-badf4596-4243-48bc-8058-0f02f6d61b2f None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:50:59.952737 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-badf4596-4243-48bc-8058-0f02f6d61b2f None None] PATH_INFO: `/` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:50:59.954068 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 24/47] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 26 10:50:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 10:50:59.960296 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca27f8b-eda8-4608-ba6b-3da3b96d4b65 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:50:59.960675 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca27f8b-eda8-4608-ba6b-3da3b96d4b65 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:50:59.960853 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca27f8b-eda8-4608-ba6b-3da3b96d4b65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:50:59.979705 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-5ca27f8b-eda8-4608-ba6b-3da3b96d4b65 None None] MFA Rules not processed for user `ca155a48e2024c32af3fd2e19e5fac11`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 10:50:59.981214 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-5ca27f8b-eda8-4608-ba6b-3da3b96d4b65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 10:51:00.007752 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:51:00.049455 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 24/48] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 26 10:50:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 10:51:00.321938 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-badf4596-4243-48bc-8058-0f02f6d61b2f None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:51:00.424737 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-badf4596-4243-48bc-8058-0f02f6d61b2f None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:51:00.460636 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a5c1db-1004-416c-8991-80fab0149be0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:51:00.461272 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a5c1db-1004-416c-8991-80fab0149be0 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:51:00.461555 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a5c1db-1004-416c-8991-80fab0149be0 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:51:00.461824 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a5c1db-1004-416c-8991-80fab0149be0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:51:00.472935 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a5c1db-1004-416c-8991-80fab0149be0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 10:51:00.582997 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application [None req-71a5c1db-1004-416c-8991-80fab0149be0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 26 10:51:00.582997 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 10:51:00.582997 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 10:51:00.582997 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 10:51:00.582997 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 10:51:00.582997 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 10:51:00.582997 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 10:51:00.582997 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 10:51:00.582997 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 10:51:00.582997 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 10:51:00.582997 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 10:51:00.582997 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 10:51:00.582997 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 10:51:00.582997 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 10:51:00.582997 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 10:51:00.582997 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 10:51:00.584001 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 10:51:00.584001 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 10:51:00.584001 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 10:51:00.584001 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 10:51:00.584001 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 10:51:00.584001 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 10:51:00.584001 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 10:51:00.584001 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 10:51:00.584001 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 10:51:00.584001 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 10:51:00.584001 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 10:51:00.584001 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application with Lock( Aug 26 10:51:00.584001 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 10:51:00.584001 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._enter() Aug 26 10:51:00.584001 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 10:51:00.584001 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 10:51:00.584747 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 10:51:00.584747 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.creator() Aug 26 10:51:00.584747 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 10:51:00.584747 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application created_value = creator( Aug 26 10:51:00.584747 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 10:51:00.584747 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 10:51:00.584747 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 10:51:00.584747 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 10:51:00.584747 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 10:51:00.584747 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 10:51:00.584747 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 26 10:51:00.584747 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Aug 26 10:51:00.584747 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 25/49] 10.222.0.40 () {56 vars in 1096 bytes} [Fri Aug 26 10:51:00 2022] GET /identity/v3/roles/admin => generated 83 bytes in 263 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 10:51:00.589994 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca27f8b-eda8-4608-ba6b-3da3b96d4b65 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:51:00.596498 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464c5afa-6544-42cf-8666-15da7da0b9ad None None] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:51:00.596825 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464c5afa-6544-42cf-8666-15da7da0b9ad None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:51:00.597198 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464c5afa-6544-42cf-8666-15da7da0b9ad None None] PATH_INFO: `/` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:51:00.598558 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 26/50] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 26 10:51:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 26 10:51:00.603405 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fc046c-811d-472d-8247-95818f4058f0 None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:51:00.603687 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fc046c-811d-472d-8247-95818f4058f0 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:51:00.603956 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fc046c-811d-472d-8247-95818f4058f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:51:00.622767 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-62fc046c-811d-472d-8247-95818f4058f0 None None] MFA Rules not processed for user `ca155a48e2024c32af3fd2e19e5fac11`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 10:51:00.624305 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-62fc046c-811d-472d-8247-95818f4058f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 10:51:00.668247 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:51:00.688540 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca27f8b-eda8-4608-ba6b-3da3b96d4b65 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:51:00.693346 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 27/51] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 26 10:51:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 26 10:51:00.721101 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b50c1d0-cbc1-4be3-ac8d-805c51319b3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:51:00.721790 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b50c1d0-cbc1-4be3-ac8d-805c51319b3f None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:51:00.722097 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b50c1d0-cbc1-4be3-ac8d-805c51319b3f None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:51:00.722371 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b50c1d0-cbc1-4be3-ac8d-805c51319b3f None admin] PATH_INFO: `/v3/roles` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:51:00.723030 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b50c1d0-cbc1-4be3-ac8d-805c51319b3f None admin] RBAC: Adding query filter params (name=admin) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 10:51:00.723329 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b50c1d0-cbc1-4be3-ac8d-805c51319b3f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:51:00.724841 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b50c1d0-cbc1-4be3-ac8d-805c51319b3f None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:51:00.732851 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 25/52] 10.222.0.40 () {56 vars in 1099 bytes} [Fri Aug 26 10:51:00 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 10:51:00.739349 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62fc046c-811d-472d-8247-95818f4058f0 None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:51:00.835545 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62fc046c-811d-472d-8247-95818f4058f0 None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:51:00.869354 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee50c5f-51fe-45d6-a7ce-be336a6d45e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:51:00.870020 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee50c5f-51fe-45d6-a7ce-be336a6d45e1 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:51:00.870299 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee50c5f-51fe-45d6-a7ce-be336a6d45e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:51:00.870575 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee50c5f-51fe-45d6-a7ce-be336a6d45e1 None admin] PATH_INFO: `/v3/users/ca155a48e2024c32af3fd2e19e5fac11` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:51:00.871715 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cee50c5f-51fe-45d6-a7ce-be336a6d45e1 None admin] RBAC: Authorizing `identity:get_user(user_id=ca155a48e2024c32af3fd2e19e5fac11)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:51:00.873160 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cee50c5f-51fe-45d6-a7ce-be336a6d45e1 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:51:00.874750 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 28/53] 10.222.0.40 () {56 vars in 1177 bytes} [Fri Aug 26 10:51:00 2022] GET /identity/v3/users/ca155a48e2024c32af3fd2e19e5fac11 => generated 247 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 10:51:00.880563 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b50c1d0-cbc1-4be3-ac8d-805c51319b3f None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:51:00.907643 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:51:00.968442 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee50c5f-51fe-45d6-a7ce-be336a6d45e1 None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:51:00.980407 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b50c1d0-cbc1-4be3-ac8d-805c51319b3f None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:51:01.016293 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:51:01.016461 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed67d280-696c-4fdd-9180-468db315b879 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:51:01.016939 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed67d280-696c-4fdd-9180-468db315b879 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:51:01.017217 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed67d280-696c-4fdd-9180-468db315b879 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:51:01.017661 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed67d280-696c-4fdd-9180-468db315b879 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:51:01.019066 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed67d280-696c-4fdd-9180-468db315b879 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:51:01.020521 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed67d280-696c-4fdd-9180-468db315b879 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:51:01.022235 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 26/54] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Aug 26 10:51:00 2022] GET /identity/v3/domains/default => generated 199 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 10:51:01.027928 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed67d280-696c-4fdd-9180-468db315b879 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:51:01.070790 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee50c5f-51fe-45d6-a7ce-be336a6d45e1 None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:51:01.107324 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f92497-e3e6-4a58-8c71-ee92c0ec8193 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:51:01.108146 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f92497-e3e6-4a58-8c71-ee92c0ec8193 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:51:01.108428 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f92497-e3e6-4a58-8c71-ee92c0ec8193 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:51:01.108709 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f92497-e3e6-4a58-8c71-ee92c0ec8193 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:51:01.109848 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f92497-e3e6-4a58-8c71-ee92c0ec8193 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:51:01.111339 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f92497-e3e6-4a58-8c71-ee92c0ec8193 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:51:01.113717 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 29/55] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Aug 26 10:51:00 2022] GET /identity/v3/domains/default => generated 199 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 10:51:01.119160 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f92497-e3e6-4a58-8c71-ee92c0ec8193 None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:51:01.120157 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:51:01.128263 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed67d280-696c-4fdd-9180-468db315b879 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:51:01.162584 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e48f2b-45ed-45b5-ae4a-3d45f64b949c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:51:01.163189 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e48f2b-45ed-45b5-ae4a-3d45f64b949c None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:51:01.163472 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e48f2b-45ed-45b5-ae4a-3d45f64b949c None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:51:01.163757 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e48f2b-45ed-45b5-ae4a-3d45f64b949c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:51:01.164480 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e48f2b-45ed-45b5-ae4a-3d45f64b949c None admin] RBAC: Adding query filter params (role.id=1f97e4602e2e4d6498913a81f9ae8d6a, scope.domain.id=default, user.id=ca155a48e2024c32af3fd2e19e5fac11) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 10:51:01.164790 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e48f2b-45ed-45b5-ae4a-3d45f64b949c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:51:01.166285 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e48f2b-45ed-45b5-ae4a-3d45f64b949c None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:51:01.171920 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 27/56] 10.222.0.40 () {56 vars in 1322 bytes} [Fri Aug 26 10:51:01 2022] GET /identity/v3/role_assignments?user.id=ca155a48e2024c32af3fd2e19e5fac11&scope.domain.id=default&role.id=1f97e4602e2e4d6498913a81f9ae8d6a => generated 530 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 10:51:01.224952 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f92497-e3e6-4a58-8c71-ee92c0ec8193 None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:51:01.240160 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:51:01.261395 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4a9278-aa88-44a2-8a2e-1193c33fb97b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:51:01.262051 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4a9278-aa88-44a2-8a2e-1193c33fb97b None admin] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:51:01.262335 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4a9278-aa88-44a2-8a2e-1193c33fb97b None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:51:01.262610 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4a9278-aa88-44a2-8a2e-1193c33fb97b None admin] PATH_INFO: `/v3/projects` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:51:01.263268 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee4a9278-aa88-44a2-8a2e-1193c33fb97b None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:51:01.264734 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee4a9278-aa88-44a2-8a2e-1193c33fb97b None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:51:01.294505 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 30/57] 10.222.0.40 () {60 vars in 1140 bytes} [Fri Aug 26 10:51:01 2022] POST /identity/v3/projects => generated 298 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 26 10:51:03.194404 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542c2660-788c-4499-b860-c8d358e33c2a None None] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:51:03.194902 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542c2660-788c-4499-b860-c8d358e33c2a None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:51:03.194902 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542c2660-788c-4499-b860-c8d358e33c2a None None] PATH_INFO: `/` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:51:03.196208 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 28/58] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 26 10:51:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 10:51:03.201903 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f2a02f-0cb3-4179-a6e4-aa7939b71b9a None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:51:03.202286 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f2a02f-0cb3-4179-a6e4-aa7939b71b9a None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:51:03.202496 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f2a02f-0cb3-4179-a6e4-aa7939b71b9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:51:03.220160 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-a3f2a02f-0cb3-4179-a6e4-aa7939b71b9a None None] MFA Rules not processed for user `ca155a48e2024c32af3fd2e19e5fac11`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 10:51:03.221365 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-a3f2a02f-0cb3-4179-a6e4-aa7939b71b9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 10:51:03.287726 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 31/59] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 26 10:51:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 10:51:03.562908 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542c2660-788c-4499-b860-c8d358e33c2a None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:51:03.662482 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542c2660-788c-4499-b860-c8d358e33c2a None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:51:03.694491 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aefc5965-77b5-45db-a15c-0bbb1de259a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:51:03.695105 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefc5965-77b5-45db-a15c-0bbb1de259a0 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:51:03.695382 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefc5965-77b5-45db-a15c-0bbb1de259a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:51:03.695651 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefc5965-77b5-45db-a15c-0bbb1de259a0 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:51:03.696743 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aefc5965-77b5-45db-a15c-0bbb1de259a0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:51:03.698214 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aefc5965-77b5-45db-a15c-0bbb1de259a0 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:51:03.699751 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 29/60] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Aug 26 10:51:03 2022] GET /identity/v3/domains/default => generated 199 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 10:51:03.705298 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f2a02f-0cb3-4179-a6e4-aa7939b71b9a None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:51:03.796671 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:51:03.802968 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f2a02f-0cb3-4179-a6e4-aa7939b71b9a None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:51:03.837327 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ec7000-edc5-4af2-b720-04c8e91c32e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:51:03.837876 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ec7000-edc5-4af2-b720-04c8e91c32e1 None admin] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:51:03.838231 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ec7000-edc5-4af2-b720-04c8e91c32e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:51:03.838506 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ec7000-edc5-4af2-b720-04c8e91c32e1 None admin] PATH_INFO: `/v3/users` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:51:03.839148 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ec7000-edc5-4af2-b720-04c8e91c32e1 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:51:03.840582 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ec7000-edc5-4af2-b720-04c8e91c32e1 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:51:03.866840 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 32/61] 10.222.0.40 () {60 vars in 1132 bytes} [Fri Aug 26 10:51:03 2022] POST /identity/v3/users => generated 275 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 10:51:06.978128 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19a2b28-5a26-4b11-bfc9-6699bc794298 None None] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:51:06.979111 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19a2b28-5a26-4b11-bfc9-6699bc794298 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:51:06.979601 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19a2b28-5a26-4b11-bfc9-6699bc794298 None None] PATH_INFO: `/` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:51:06.981432 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 30/62] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 26 10:51:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 10:51:06.986681 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5da6e53-6a41-4d0f-8cc3-67888428cf9a None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:51:06.987598 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5da6e53-6a41-4d0f-8cc3-67888428cf9a None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:51:06.988144 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5da6e53-6a41-4d0f-8cc3-67888428cf9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:51:06.993052 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b22da5-9a49-4f5b-a1f4-1519011a3ca1 None None] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:51:06.993566 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b22da5-9a49-4f5b-a1f4-1519011a3ca1 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:51:06.994114 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b22da5-9a49-4f5b-a1f4-1519011a3ca1 None None] PATH_INFO: `/` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:51:06.995657 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 31/63] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 26 10:51:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 26 10:51:07.000492 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc82fcdc-67ba-4421-93d7-6cb6c88e1f12 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:51:07.001006 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc82fcdc-67ba-4421-93d7-6cb6c88e1f12 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:51:07.001485 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc82fcdc-67ba-4421-93d7-6cb6c88e1f12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:51:07.026613 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-e5da6e53-6a41-4d0f-8cc3-67888428cf9a None None] MFA Rules not processed for user `ca155a48e2024c32af3fd2e19e5fac11`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 10:51:07.028362 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-e5da6e53-6a41-4d0f-8cc3-67888428cf9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 10:51:07.034237 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-bc82fcdc-67ba-4421-93d7-6cb6c88e1f12 None None] MFA Rules not processed for user `ca155a48e2024c32af3fd2e19e5fac11`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 10:51:07.036729 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-bc82fcdc-67ba-4421-93d7-6cb6c88e1f12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 10:51:07.073271 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:51:07.119508 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 33/64] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 26 10:51:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 10:51:07.121288 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3338da-32b8-4007-ab0b-c899af8b341a None None] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:51:07.121853 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3338da-32b8-4007-ab0b-c899af8b341a None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:51:07.122381 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3338da-32b8-4007-ab0b-c899af8b341a None None] PATH_INFO: `/` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:51:07.124098 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 34/65] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 26 10:51:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 10:51:07.132185 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 32/66] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 26 10:51:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 10:51:07.133760 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c0cc4f-be6a-4f9d-a2b3-bb3bb0d927ab None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:51:07.134308 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c0cc4f-be6a-4f9d-a2b3-bb3bb0d927ab None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:51:07.134788 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c0cc4f-be6a-4f9d-a2b3-bb3bb0d927ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:51:07.161343 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-89c0cc4f-be6a-4f9d-a2b3-bb3bb0d927ab None None] MFA Rules not processed for user `ca155a48e2024c32af3fd2e19e5fac11`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 10:51:07.162967 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-89c0cc4f-be6a-4f9d-a2b3-bb3bb0d927ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 10:51:07.234753 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 35/67] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 26 10:51:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 10:51:07.392113 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc82fcdc-67ba-4421-93d7-6cb6c88e1f12 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:51:07.496645 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c0cc4f-be6a-4f9d-a2b3-bb3bb0d927ab None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:51:07.518248 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:51:07.519607 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc82fcdc-67ba-4421-93d7-6cb6c88e1f12 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:51:07.557660 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936aece4-c16d-4825-ac0f-5e6448b7f17f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:51:07.558335 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936aece4-c16d-4825-ac0f-5e6448b7f17f None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:51:07.558612 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936aece4-c16d-4825-ac0f-5e6448b7f17f None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:51:07.558924 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936aece4-c16d-4825-ac0f-5e6448b7f17f None admin] PATH_INFO: `/v3/roles/member` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:51:07.569920 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-936aece4-c16d-4825-ac0f-5e6448b7f17f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 10:51:07.574745 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application [None req-936aece4-c16d-4825-ac0f-5e6448b7f17f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 26 10:51:07.574745 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 10:51:07.574745 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 10:51:07.574745 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 10:51:07.574745 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 10:51:07.574745 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 10:51:07.574745 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 10:51:07.574745 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 10:51:07.574745 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 10:51:07.574745 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 10:51:07.574745 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 10:51:07.574745 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 10:51:07.574745 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 10:51:07.574745 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 10:51:07.574745 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 10:51:07.574745 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 10:51:07.575637 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 10:51:07.575637 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 10:51:07.575637 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 10:51:07.575637 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 10:51:07.575637 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 10:51:07.575637 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 10:51:07.575637 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 10:51:07.575637 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 10:51:07.575637 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 10:51:07.575637 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 10:51:07.575637 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 10:51:07.575637 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application with Lock( Aug 26 10:51:07.575637 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 10:51:07.575637 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._enter() Aug 26 10:51:07.575637 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 10:51:07.575637 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 10:51:07.578839 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 10:51:07.578839 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.creator() Aug 26 10:51:07.578839 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 10:51:07.578839 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application created_value = creator( Aug 26 10:51:07.578839 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 10:51:07.578839 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 10:51:07.578839 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 10:51:07.578839 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 10:51:07.578839 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 10:51:07.578839 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 10:51:07.578839 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 26 10:51:07.578839 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Aug 26 10:51:07.578839 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 33/68] 10.222.0.40 () {56 vars in 1099 bytes} [Fri Aug 26 10:51:07 2022] GET /identity/v3/roles/member => generated 84 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 10:51:07.578839 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778abb31-716e-449a-aa06-c6d2b2e8e6c6 None None] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:51:07.579513 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778abb31-716e-449a-aa06-c6d2b2e8e6c6 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:51:07.579513 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778abb31-716e-449a-aa06-c6d2b2e8e6c6 None None] PATH_INFO: `/` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:51:07.579965 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 34/69] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 26 10:51:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 10:51:07.580807 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aeb4504-db4b-408c-8c54-e8eee810d76f None None] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:51:07.581261 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aeb4504-db4b-408c-8c54-e8eee810d76f None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:51:07.581614 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aeb4504-db4b-408c-8c54-e8eee810d76f None None] PATH_INFO: `/` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:51:07.582940 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 35/70] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 26 10:51:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 10:51:07.587626 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aeb4504-db4b-408c-8c54-e8eee810d76f None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:51:07.608132 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:51:07.627854 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c0cc4f-be6a-4f9d-a2b3-bb3bb0d927ab None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:51:07.662303 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1961a459-2f88-438a-a9d0-713df6f1e071 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:51:07.662882 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1961a459-2f88-438a-a9d0-713df6f1e071 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:51:07.663164 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1961a459-2f88-438a-a9d0-713df6f1e071 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:51:07.663437 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1961a459-2f88-438a-a9d0-713df6f1e071 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:51:07.673360 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1961a459-2f88-438a-a9d0-713df6f1e071 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 10:51:07.678841 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application [None req-1961a459-2f88-438a-a9d0-713df6f1e071 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 26 10:51:07.678841 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 10:51:07.678841 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 10:51:07.678841 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 10:51:07.678841 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 10:51:07.678841 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 10:51:07.678841 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 10:51:07.678841 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 10:51:07.678841 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 10:51:07.678841 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 10:51:07.678841 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 10:51:07.678841 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 10:51:07.678841 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 10:51:07.678841 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 10:51:07.678841 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 10:51:07.678841 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 10:51:07.679701 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 10:51:07.679701 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 10:51:07.679701 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 10:51:07.679701 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 10:51:07.679701 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 10:51:07.679701 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 10:51:07.679701 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 10:51:07.679701 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 10:51:07.679701 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 10:51:07.679701 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 10:51:07.679701 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 10:51:07.679701 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application with Lock( Aug 26 10:51:07.679701 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 10:51:07.679701 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._enter() Aug 26 10:51:07.679701 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 10:51:07.679701 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 10:51:07.680472 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 10:51:07.680472 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.creator() Aug 26 10:51:07.680472 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 10:51:07.680472 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application created_value = creator( Aug 26 10:51:07.680472 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 10:51:07.680472 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 10:51:07.680472 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 10:51:07.680472 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 10:51:07.680472 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 10:51:07.680472 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 10:51:07.680472 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 26 10:51:07.680472 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Aug 26 10:51:07.680472 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 36/71] 10.222.0.40 () {56 vars in 1096 bytes} [Fri Aug 26 10:51:07 2022] GET /identity/v3/roles/admin => generated 83 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 10:51:07.682816 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6256f7-250e-49f9-9b17-7d04b9d474bb None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:51:07.683110 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6256f7-250e-49f9-9b17-7d04b9d474bb None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:51:07.683387 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6256f7-250e-49f9-9b17-7d04b9d474bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:51:07.693602 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aeb4504-db4b-408c-8c54-e8eee810d76f None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:51:07.700170 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-0f6256f7-250e-49f9-9b17-7d04b9d474bb None None] MFA Rules not processed for user `ca155a48e2024c32af3fd2e19e5fac11`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 10:51:07.701406 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-0f6256f7-250e-49f9-9b17-7d04b9d474bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 10:51:07.727594 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009cee26-ea6e-48a6-9b9f-6ccca71fe47d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:51:07.728206 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009cee26-ea6e-48a6-9b9f-6ccca71fe47d None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:51:07.728487 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009cee26-ea6e-48a6-9b9f-6ccca71fe47d None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:51:07.728766 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009cee26-ea6e-48a6-9b9f-6ccca71fe47d None admin] PATH_INFO: `/v3/roles` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:51:07.729427 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-009cee26-ea6e-48a6-9b9f-6ccca71fe47d None admin] RBAC: Adding query filter params (name=member) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 10:51:07.729730 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-009cee26-ea6e-48a6-9b9f-6ccca71fe47d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:51:07.731182 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-009cee26-ea6e-48a6-9b9f-6ccca71fe47d None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:51:07.739975 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 36/72] 10.222.0.40 () {56 vars in 1101 bytes} [Fri Aug 26 10:51:07 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 10:51:07.741118 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ded4c2-64bd-4b88-8a3a-e8f7392f4ca1 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:51:07.741403 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ded4c2-64bd-4b88-8a3a-e8f7392f4ca1 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:51:07.741682 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ded4c2-64bd-4b88-8a3a-e8f7392f4ca1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:51:07.758720 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-17ded4c2-64bd-4b88-8a3a-e8f7392f4ca1 None None] MFA Rules not processed for user `ca155a48e2024c32af3fd2e19e5fac11`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 10:51:07.759956 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-17ded4c2-64bd-4b88-8a3a-e8f7392f4ca1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 10:51:07.773810 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 37/73] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 26 10:51:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 10:51:07.775034 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6256f7-250e-49f9-9b17-7d04b9d474bb None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:51:07.776906 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:51:07.834772 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 37/74] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 26 10:51:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 10:51:07.838195 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ded4c2-64bd-4b88-8a3a-e8f7392f4ca1 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:51:07.846164 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:51:07.896610 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6256f7-250e-49f9-9b17-7d04b9d474bb None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:51:07.935997 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f70db6-d35f-4caf-879f-dca95831d212 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:51:07.937030 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f70db6-d35f-4caf-879f-dca95831d212 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:51:07.937481 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f70db6-d35f-4caf-879f-dca95831d212 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:51:07.937919 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f70db6-d35f-4caf-879f-dca95831d212 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:51:07.949489 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f70db6-d35f-4caf-879f-dca95831d212 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 10:51:07.954768 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application [None req-69f70db6-d35f-4caf-879f-dca95831d212 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 26 10:51:07.954768 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 10:51:07.954768 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 10:51:07.954768 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 10:51:07.954768 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 10:51:07.954768 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 10:51:07.954768 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 10:51:07.954768 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 10:51:07.954768 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 10:51:07.954768 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 10:51:07.954768 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 10:51:07.954768 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 10:51:07.954768 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 10:51:07.954768 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 10:51:07.954768 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 10:51:07.954768 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 10:51:07.955612 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 10:51:07.955612 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 10:51:07.955612 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 10:51:07.955612 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 10:51:07.955612 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 10:51:07.955612 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 10:51:07.955612 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 10:51:07.955612 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 10:51:07.955612 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 10:51:07.955612 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 10:51:07.955612 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 10:51:07.955612 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application with Lock( Aug 26 10:51:07.955612 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 10:51:07.955612 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._enter() Aug 26 10:51:07.955612 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 10:51:07.955612 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 10:51:07.956982 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 10:51:07.956982 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.creator() Aug 26 10:51:07.956982 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 10:51:07.956982 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application created_value = creator( Aug 26 10:51:07.956982 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 10:51:07.956982 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 10:51:07.956982 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 10:51:07.956982 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 10:51:07.956982 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 10:51:07.956982 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 10:51:07.956982 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 26 10:51:07.956982 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Aug 26 10:51:07.957544 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ded4c2-64bd-4b88-8a3a-e8f7392f4ca1 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:51:07.959605 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 38/75] 10.222.0.40 () {56 vars in 1114 bytes} [Fri Aug 26 10:51:07 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 10:51:07.960697 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f70db6-d35f-4caf-879f-dca95831d212 None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:51:08.003863 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2d52c2-92f0-4b76-96f5-0e8557ca0397 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:51:08.005130 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2d52c2-92f0-4b76-96f5-0e8557ca0397 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:51:08.005548 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2d52c2-92f0-4b76-96f5-0e8557ca0397 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:51:08.005963 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2d52c2-92f0-4b76-96f5-0e8557ca0397 None admin] PATH_INFO: `/v3/roles` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:51:08.007068 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c2d52c2-92f0-4b76-96f5-0e8557ca0397 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 10:51:08.007548 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c2d52c2-92f0-4b76-96f5-0e8557ca0397 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:51:08.009872 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c2d52c2-92f0-4b76-96f5-0e8557ca0397 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:51:08.020940 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 38/76] 10.222.0.40 () {56 vars in 1099 bytes} [Fri Aug 26 10:51:07 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 10:51:08.022062 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2d52c2-92f0-4b76-96f5-0e8557ca0397 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:51:08.046335 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:51:08.086521 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f70db6-d35f-4caf-879f-dca95831d212 None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:51:08.101747 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:51:08.121148 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8764006-a2dd-4d75-a528-8af45cd2f3e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:51:08.121783 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8764006-a2dd-4d75-a528-8af45cd2f3e7 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:51:08.122107 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8764006-a2dd-4d75-a528-8af45cd2f3e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:51:08.122385 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8764006-a2dd-4d75-a528-8af45cd2f3e7 None admin] PATH_INFO: `/v3/users/514ea0d902e04c41a548d79008a6d68d` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:51:08.134104 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2d52c2-92f0-4b76-96f5-0e8557ca0397 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:51:08.137250 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8764006-a2dd-4d75-a528-8af45cd2f3e7 None admin] RBAC: Authorizing `identity:get_user(user_id=514ea0d902e04c41a548d79008a6d68d)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:51:08.138773 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8764006-a2dd-4d75-a528-8af45cd2f3e7 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:51:08.140685 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 39/77] 10.222.0.40 () {56 vars in 1177 bytes} [Fri Aug 26 10:51:07 2022] GET /identity/v3/users/514ea0d902e04c41a548d79008a6d68d => generated 275 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 10:51:08.141355 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8764006-a2dd-4d75-a528-8af45cd2f3e7 None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:51:08.169031 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1fa3f4-6651-4738-874a-8094193e0b4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:51:08.169641 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1fa3f4-6651-4738-874a-8094193e0b4d None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:51:08.169919 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1fa3f4-6651-4738-874a-8094193e0b4d None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:51:08.170212 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1fa3f4-6651-4738-874a-8094193e0b4d None admin] PATH_INFO: `/v3/roles` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:51:08.170882 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b1fa3f4-6651-4738-874a-8094193e0b4d None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 10:51:08.171183 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b1fa3f4-6651-4738-874a-8094193e0b4d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:51:08.172568 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b1fa3f4-6651-4738-874a-8094193e0b4d None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:51:08.180581 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 39/78] 10.222.0.40 () {56 vars in 1111 bytes} [Fri Aug 26 10:51:08 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 10:51:08.181374 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1fa3f4-6651-4738-874a-8094193e0b4d None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:51:08.245242 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8764006-a2dd-4d75-a528-8af45cd2f3e7 None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:51:08.264406 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:51:08.279326 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3216465c-2ddf-4614-b88b-e23215e610c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:51:08.279960 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3216465c-2ddf-4614-b88b-e23215e610c7 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:51:08.280241 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3216465c-2ddf-4614-b88b-e23215e610c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:51:08.280454 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1fa3f4-6651-4738-874a-8094193e0b4d None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:51:08.280550 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3216465c-2ddf-4614-b88b-e23215e610c7 None admin] PATH_INFO: `/v3/users/ca155a48e2024c32af3fd2e19e5fac11` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:51:08.281646 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3216465c-2ddf-4614-b88b-e23215e610c7 None admin] RBAC: Authorizing `identity:get_user(user_id=ca155a48e2024c32af3fd2e19e5fac11)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:51:08.283136 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3216465c-2ddf-4614-b88b-e23215e610c7 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:51:08.285803 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 40/79] 10.222.0.40 () {56 vars in 1177 bytes} [Fri Aug 26 10:51:08 2022] GET /identity/v3/users/ca155a48e2024c32af3fd2e19e5fac11 => generated 247 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 10:51:08.286606 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3216465c-2ddf-4614-b88b-e23215e610c7 None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:51:08.309813 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:51:08.315221 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc0ce8b-e379-4c5f-97ea-0b1805fe4dbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:51:08.316542 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc0ce8b-e379-4c5f-97ea-0b1805fe4dbf None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:51:08.316826 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc0ce8b-e379-4c5f-97ea-0b1805fe4dbf None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:51:08.317100 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc0ce8b-e379-4c5f-97ea-0b1805fe4dbf None admin] PATH_INFO: `/v3/roles/member` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:51:08.327063 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc0ce8b-e379-4c5f-97ea-0b1805fe4dbf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 10:51:08.332779 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application [None req-3bc0ce8b-e379-4c5f-97ea-0b1805fe4dbf None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 26 10:51:08.332779 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 10:51:08.332779 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 10:51:08.332779 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 10:51:08.332779 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 10:51:08.332779 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 10:51:08.332779 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 10:51:08.332779 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 10:51:08.332779 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 10:51:08.332779 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 10:51:08.332779 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 10:51:08.332779 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 10:51:08.332779 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 10:51:08.332779 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 10:51:08.332779 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 10:51:08.332779 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 10:51:08.333606 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 10:51:08.333606 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 10:51:08.333606 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 10:51:08.333606 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 10:51:08.333606 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 10:51:08.333606 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 10:51:08.333606 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 10:51:08.333606 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 10:51:08.333606 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 10:51:08.333606 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 10:51:08.333606 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 10:51:08.333606 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application with Lock( Aug 26 10:51:08.333606 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 10:51:08.333606 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._enter() Aug 26 10:51:08.333606 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 10:51:08.333606 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 10:51:08.334690 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 10:51:08.334690 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.creator() Aug 26 10:51:08.334690 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 10:51:08.334690 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application created_value = creator( Aug 26 10:51:08.334690 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 10:51:08.334690 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 10:51:08.334690 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 10:51:08.334690 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 10:51:08.334690 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 10:51:08.334690 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 10:51:08.334690 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 26 10:51:08.334690 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Aug 26 10:51:08.334690 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 40/80] 10.222.0.40 () {56 vars in 1099 bytes} [Fri Aug 26 10:51:08 2022] GET /identity/v3/roles/member => generated 84 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 10:51:08.334690 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc0ce8b-e379-4c5f-97ea-0b1805fe4dbf None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:51:08.391465 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3216465c-2ddf-4614-b88b-e23215e610c7 None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:51:08.424899 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5727104-5626-489e-8d6d-3664cdf9f40c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:51:08.425808 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5727104-5626-489e-8d6d-3664cdf9f40c None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:51:08.426123 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5727104-5626-489e-8d6d-3664cdf9f40c None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:51:08.426403 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5727104-5626-489e-8d6d-3664cdf9f40c None admin] PATH_INFO: `/v3/domains/default` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:51:08.427533 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5727104-5626-489e-8d6d-3664cdf9f40c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:51:08.428978 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5727104-5626-489e-8d6d-3664cdf9f40c None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:51:08.430577 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 41/81] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Aug 26 10:51:08 2022] GET /identity/v3/domains/default => generated 199 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 10:51:08.430577 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5727104-5626-489e-8d6d-3664cdf9f40c None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:51:08.437837 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc0ce8b-e379-4c5f-97ea-0b1805fe4dbf None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:51:08.471207 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ac469e-8f66-4b7a-ae08-bf83ef2f908e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:51:08.471844 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ac469e-8f66-4b7a-ae08-bf83ef2f908e None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:51:08.472121 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ac469e-8f66-4b7a-ae08-bf83ef2f908e None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:51:08.472387 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ac469e-8f66-4b7a-ae08-bf83ef2f908e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:51:08.487573 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:51:08.530229 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:51:08.532452 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5727104-5626-489e-8d6d-3664cdf9f40c None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:51:08.540460 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ac469e-8f66-4b7a-ae08-bf83ef2f908e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:51:08.541938 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ac469e-8f66-4b7a-ae08-bf83ef2f908e None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:51:08.566118 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd49c15-023e-4d8b-93e6-dad71d5edb00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:51:08.566714 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd49c15-023e-4d8b-93e6-dad71d5edb00 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:51:08.566993 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd49c15-023e-4d8b-93e6-dad71d5edb00 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:51:08.567273 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd49c15-023e-4d8b-93e6-dad71d5edb00 None admin] PATH_INFO: `/v3/users/514ea0d902e04c41a548d79008a6d68d` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:51:08.568418 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd49c15-023e-4d8b-93e6-dad71d5edb00 None admin] RBAC: Authorizing `identity:get_user(user_id=514ea0d902e04c41a548d79008a6d68d)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:51:08.569846 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd49c15-023e-4d8b-93e6-dad71d5edb00 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:51:08.571448 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 42/82] 10.222.0.40 () {56 vars in 1177 bytes} [Fri Aug 26 10:51:08 2022] GET /identity/v3/users/514ea0d902e04c41a548d79008a6d68d => generated 275 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 10:51:08.572215 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd49c15-023e-4d8b-93e6-dad71d5edb00 None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:51:08.642939 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 41/83] 10.222.0.40 () {58 vars in 1282 bytes} [Fri Aug 26 10:51:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 26 10:51:08.644357 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ac469e-8f66-4b7a-ae08-bf83ef2f908e None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:51:08.672571 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd49c15-023e-4d8b-93e6-dad71d5edb00 None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:51:08.696622 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:51:08.708695 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f17c2e2-87b7-4fd3-b2c2-97ec0592d95d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:51:08.709307 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f17c2e2-87b7-4fd3-b2c2-97ec0592d95d None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:51:08.709590 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f17c2e2-87b7-4fd3-b2c2-97ec0592d95d None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:51:08.709866 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f17c2e2-87b7-4fd3-b2c2-97ec0592d95d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:51:08.734353 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:51:08.744472 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ac469e-8f66-4b7a-ae08-bf83ef2f908e None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:51:08.778170 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c5397d-4c7c-4f8a-8b90-f265f4267e92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:51:08.778777 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c5397d-4c7c-4f8a-8b90-f265f4267e92 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:51:08.778873 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f17c2e2-87b7-4fd3-b2c2-97ec0592d95d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:51:08.779058 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c5397d-4c7c-4f8a-8b90-f265f4267e92 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:51:08.779331 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c5397d-4c7c-4f8a-8b90-f265f4267e92 None admin] PATH_INFO: `/v3/roles` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:51:08.779989 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c5397d-4c7c-4f8a-8b90-f265f4267e92 None admin] RBAC: Adding query filter params (name=member) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 10:51:08.780342 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c5397d-4c7c-4f8a-8b90-f265f4267e92 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:51:08.780439 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f17c2e2-87b7-4fd3-b2c2-97ec0592d95d None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:51:08.781654 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c5397d-4c7c-4f8a-8b90-f265f4267e92 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:51:08.790459 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 42/84] 10.222.0.40 () {56 vars in 1101 bytes} [Fri Aug 26 10:51:08 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 10:51:08.791234 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c5397d-4c7c-4f8a-8b90-f265f4267e92 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:51:08.887885 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 43/85] 10.222.0.40 () {58 vars in 1282 bytes} [Fri Aug 26 10:51:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 26 10:51:08.888733 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f17c2e2-87b7-4fd3-b2c2-97ec0592d95d None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:51:08.896058 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c5397d-4c7c-4f8a-8b90-f265f4267e92 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:51:08.913086 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:51:08.929627 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a831b71a-7a96-4295-a3da-583eecdecfec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:51:08.930261 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a831b71a-7a96-4295-a3da-583eecdecfec None admin] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:51:08.930539 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a831b71a-7a96-4295-a3da-583eecdecfec None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:51:08.930815 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a831b71a-7a96-4295-a3da-583eecdecfec None admin] PATH_INFO: `/v3/users` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:51:08.931452 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a831b71a-7a96-4295-a3da-583eecdecfec None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:51:08.932904 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a831b71a-7a96-4295-a3da-583eecdecfec None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:51:08.959420 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 43/86] 10.222.0.40 () {60 vars in 1132 bytes} [Fri Aug 26 10:51:08 2022] POST /identity/v3/users => generated 289 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 10:51:08.962907 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a831b71a-7a96-4295-a3da-583eecdecfec None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:51:08.986104 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:51:08.992823 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f17c2e2-87b7-4fd3-b2c2-97ec0592d95d None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:51:09.036635 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b57a08c-7e35-416d-a1d4-1f8390ad7e21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:51:09.037175 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b57a08c-7e35-416d-a1d4-1f8390ad7e21 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:51:09.037455 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b57a08c-7e35-416d-a1d4-1f8390ad7e21 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:51:09.037725 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b57a08c-7e35-416d-a1d4-1f8390ad7e21 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:51:09.087081 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a831b71a-7a96-4295-a3da-583eecdecfec None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:51:09.119416 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b57a08c-7e35-416d-a1d4-1f8390ad7e21 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:51:09.120098 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b57a08c-7e35-416d-a1d4-1f8390ad7e21 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:51:09.130589 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417ee6e9-c0f3-4ef2-acc4-c6a0323ef830 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:51:09.131530 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417ee6e9-c0f3-4ef2-acc4-c6a0323ef830 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:51:09.131617 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417ee6e9-c0f3-4ef2-acc4-c6a0323ef830 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:51:09.131891 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417ee6e9-c0f3-4ef2-acc4-c6a0323ef830 None admin] PATH_INFO: `/v3/projects/da070dea52de4d71952ed50827d44a1c` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:51:09.133155 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-417ee6e9-c0f3-4ef2-acc4-c6a0323ef830 None admin] RBAC: Authorizing `identity:get_project(project_id=da070dea52de4d71952ed50827d44a1c)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:51:09.134680 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-417ee6e9-c0f3-4ef2-acc4-c6a0323ef830 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:51:09.136445 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 44/87] 10.222.0.40 () {56 vars in 1186 bytes} [Fri Aug 26 10:51:08 2022] GET /identity/v3/projects/da070dea52de4d71952ed50827d44a1c => generated 298 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 10:51:09.137219 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417ee6e9-c0f3-4ef2-acc4-c6a0323ef830 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:51:09.147704 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:51:09.244845 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:51:09.252231 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 44/88] 10.222.0.40 () {58 vars in 1282 bytes} [Fri Aug 26 10:51:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 364 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 26 10:51:09.253861 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b57a08c-7e35-416d-a1d4-1f8390ad7e21 None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:51:09.266285 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417ee6e9-c0f3-4ef2-acc4-c6a0323ef830 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:51:09.309618 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c290f2-5bc8-476a-8058-85d5af8decdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:51:09.310314 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c290f2-5bc8-476a-8058-85d5af8decdb None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:51:09.310930 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c290f2-5bc8-476a-8058-85d5af8decdb None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:51:09.311056 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c290f2-5bc8-476a-8058-85d5af8decdb None admin] PATH_INFO: `/v3/users/514ea0d902e04c41a548d79008a6d68d` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:51:09.312120 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c290f2-5bc8-476a-8058-85d5af8decdb None admin] RBAC: Authorizing `identity:get_user(user_id=514ea0d902e04c41a548d79008a6d68d)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:51:09.313587 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c290f2-5bc8-476a-8058-85d5af8decdb None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:51:09.315647 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 45/89] 10.222.0.40 () {56 vars in 1177 bytes} [Fri Aug 26 10:51:09 2022] GET /identity/v3/users/514ea0d902e04c41a548d79008a6d68d => generated 275 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 10:51:09.316078 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c290f2-5bc8-476a-8058-85d5af8decdb None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:51:09.375394 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b57a08c-7e35-416d-a1d4-1f8390ad7e21 None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:51:09.400682 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:51:09.413879 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4c76e1-eb5f-44dd-aa9b-4cddd07e1f03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:51:09.414680 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4c76e1-eb5f-44dd-aa9b-4cddd07e1f03 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:51:09.415031 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4c76e1-eb5f-44dd-aa9b-4cddd07e1f03 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:51:09.415380 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4c76e1-eb5f-44dd-aa9b-4cddd07e1f03 None admin] PATH_INFO: `/v3/projects/da070dea52de4d71952ed50827d44a1c` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:51:09.416596 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e4c76e1-eb5f-44dd-aa9b-4cddd07e1f03 None admin] RBAC: Authorizing `identity:get_project(project_id=da070dea52de4d71952ed50827d44a1c)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:51:09.418162 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e4c76e1-eb5f-44dd-aa9b-4cddd07e1f03 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:51:09.420098 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 45/90] 10.222.0.40 () {56 vars in 1186 bytes} [Fri Aug 26 10:51:09 2022] GET /identity/v3/projects/da070dea52de4d71952ed50827d44a1c => generated 298 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 10:51:09.420779 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4c76e1-eb5f-44dd-aa9b-4cddd07e1f03 None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:51:09.449118 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c290f2-5bc8-476a-8058-85d5af8decdb None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:51:09.497226 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dafaeda9-ade6-441e-be7b-21621a27ea78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:51:09.497851 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafaeda9-ade6-441e-be7b-21621a27ea78 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:51:09.499043 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafaeda9-ade6-441e-be7b-21621a27ea78 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:51:09.499330 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafaeda9-ade6-441e-be7b-21621a27ea78 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:51:09.500338 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dafaeda9-ade6-441e-be7b-21621a27ea78 None admin] RBAC: Adding query filter params (role.id=23e174ce743b46eabe618c212691da55, scope.project.id=da070dea52de4d71952ed50827d44a1c, user.id=514ea0d902e04c41a548d79008a6d68d) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 10:51:09.500662 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dafaeda9-ade6-441e-be7b-21621a27ea78 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:51:09.502173 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dafaeda9-ade6-441e-be7b-21621a27ea78 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:51:09.509608 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 46/91] 10.222.0.40 () {56 vars in 1374 bytes} [Fri Aug 26 10:51:09 2022] GET /identity/v3/role_assignments?user.id=514ea0d902e04c41a548d79008a6d68d&scope.project.id=da070dea52de4d71952ed50827d44a1c&role.id=23e174ce743b46eabe618c212691da55 => generated 259 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 10:51:09.514375 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dafaeda9-ade6-441e-be7b-21621a27ea78 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:51:09.541703 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:51:09.552897 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4c76e1-eb5f-44dd-aa9b-4cddd07e1f03 None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:51:09.614735 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028e1240-5739-4a13-aefb-55dedf6ce090 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:51:09.616494 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028e1240-5739-4a13-aefb-55dedf6ce090 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:51:09.616779 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028e1240-5739-4a13-aefb-55dedf6ce090 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:51:09.617052 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028e1240-5739-4a13-aefb-55dedf6ce090 None admin] PATH_INFO: `/v3/projects/da070dea52de4d71952ed50827d44a1c` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:51:09.619782 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-028e1240-5739-4a13-aefb-55dedf6ce090 None admin] RBAC: Authorizing `identity:get_project(project_id=da070dea52de4d71952ed50827d44a1c)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:51:09.625136 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-028e1240-5739-4a13-aefb-55dedf6ce090 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:51:09.627284 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 46/92] 10.222.0.40 () {56 vars in 1186 bytes} [Fri Aug 26 10:51:09 2022] GET /identity/v3/projects/da070dea52de4d71952ed50827d44a1c => generated 298 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 10:51:09.628415 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028e1240-5739-4a13-aefb-55dedf6ce090 None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:51:09.714699 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dafaeda9-ade6-441e-be7b-21621a27ea78 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:51:09.739027 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:51:09.780604 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee8c034-c963-4162-b7e5-ba679044441a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:51:09.781230 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee8c034-c963-4162-b7e5-ba679044441a None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:51:09.781505 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee8c034-c963-4162-b7e5-ba679044441a None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:51:09.781774 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee8c034-c963-4162-b7e5-ba679044441a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:51:09.810013 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028e1240-5739-4a13-aefb-55dedf6ce090 None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:51:09.874379 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6681b0c1-c44d-4fe6-9e18-7a4e6f974a28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:51:09.875051 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6681b0c1-c44d-4fe6-9e18-7a4e6f974a28 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:51:09.875304 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6681b0c1-c44d-4fe6-9e18-7a4e6f974a28 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:51:09.875549 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6681b0c1-c44d-4fe6-9e18-7a4e6f974a28 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:51:09.876287 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6681b0c1-c44d-4fe6-9e18-7a4e6f974a28 None admin] RBAC: Adding query filter params (role.id=1f97e4602e2e4d6498913a81f9ae8d6a, scope.project.id=da070dea52de4d71952ed50827d44a1c, user.id=ca155a48e2024c32af3fd2e19e5fac11) {{(pid=81423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 10:51:09.876598 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6681b0c1-c44d-4fe6-9e18-7a4e6f974a28 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:51:09.878661 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6681b0c1-c44d-4fe6-9e18-7a4e6f974a28 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:51:09.891772 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 47/93] 10.222.0.40 () {56 vars in 1374 bytes} [Fri Aug 26 10:51:09 2022] GET /identity/v3/role_assignments?user.id=ca155a48e2024c32af3fd2e19e5fac11&scope.project.id=da070dea52de4d71952ed50827d44a1c&role.id=1f97e4602e2e4d6498913a81f9ae8d6a => generated 259 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 10:51:09.892619 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6681b0c1-c44d-4fe6-9e18-7a4e6f974a28 None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:51:09.905734 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee8c034-c963-4162-b7e5-ba679044441a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:51:09.907442 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee8c034-c963-4162-b7e5-ba679044441a None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:51:09.924141 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:51:10.094519 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 47/94] 10.222.0.40 () {58 vars in 1282 bytes} [Fri Aug 26 10:51:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 583 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 26 10:51:10.102615 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee8c034-c963-4162-b7e5-ba679044441a None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:51:10.168187 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6681b0c1-c44d-4fe6-9e18-7a4e6f974a28 None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:51:10.212976 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:51:10.241519 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa5afff-699c-41ae-aa42-57c6c921952a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:51:10.242208 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa5afff-699c-41ae-aa42-57c6c921952a None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:51:10.242490 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa5afff-699c-41ae-aa42-57c6c921952a None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:51:10.242773 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa5afff-699c-41ae-aa42-57c6c921952a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:51:10.243499 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa5afff-699c-41ae-aa42-57c6c921952a None admin] RBAC: Adding query filter params (role.id=9c439f898126444f92abb01910f1d02b, scope.project.id=da070dea52de4d71952ed50827d44a1c, user.id=514ea0d902e04c41a548d79008a6d68d) {{(pid=81423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 10:51:10.243806 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa5afff-699c-41ae-aa42-57c6c921952a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:51:10.245253 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa5afff-699c-41ae-aa42-57c6c921952a None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:51:10.254640 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 48/95] 10.222.0.40 () {56 vars in 1374 bytes} [Fri Aug 26 10:51:09 2022] GET /identity/v3/role_assignments?user.id=514ea0d902e04c41a548d79008a6d68d&scope.project.id=da070dea52de4d71952ed50827d44a1c&role.id=9c439f898126444f92abb01910f1d02b => generated 259 bytes in 362 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 10:51:10.294176 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee8c034-c963-4162-b7e5-ba679044441a None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:51:10.309701 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:51:10.350704 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5159d7-73c1-4613-be14-b945248e7458 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:51:10.351359 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5159d7-73c1-4613-be14-b945248e7458 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:51:10.351637 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5159d7-73c1-4613-be14-b945248e7458 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:51:10.351906 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5159d7-73c1-4613-be14-b945248e7458 None admin] PATH_INFO: `/v3/projects/f7edfd8f65e54a4683d5895b5809cedc` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:51:10.353082 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd5159d7-73c1-4613-be14-b945248e7458 None admin] RBAC: Authorizing `identity:get_project(project_id=f7edfd8f65e54a4683d5895b5809cedc)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:51:10.354575 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd5159d7-73c1-4613-be14-b945248e7458 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:51:10.356530 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 48/96] 10.222.0.40 () {56 vars in 1186 bytes} [Fri Aug 26 10:51:10 2022] GET /identity/v3/projects/f7edfd8f65e54a4683d5895b5809cedc => generated 312 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 10:51:10.363486 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa5afff-699c-41ae-aa42-57c6c921952a None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:51:10.504598 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa5afff-699c-41ae-aa42-57c6c921952a None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:51:10.556992 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9951c6-3169-4d7f-96c4-150919a9a514 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:51:10.557991 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9951c6-3169-4d7f-96c4-150919a9a514 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:51:10.558595 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9951c6-3169-4d7f-96c4-150919a9a514 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:51:10.559066 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9951c6-3169-4d7f-96c4-150919a9a514 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:51:10.559995 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e9951c6-3169-4d7f-96c4-150919a9a514 None admin] RBAC: Adding query filter params (role.id=23e174ce743b46eabe618c212691da55, scope.project.id=f7edfd8f65e54a4683d5895b5809cedc, user.id=514ea0d902e04c41a548d79008a6d68d) {{(pid=81423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 10:51:10.560498 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e9951c6-3169-4d7f-96c4-150919a9a514 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:51:10.562158 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e9951c6-3169-4d7f-96c4-150919a9a514 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:51:10.568635 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 49/97] 10.222.0.40 () {56 vars in 1374 bytes} [Fri Aug 26 10:51:10 2022] GET /identity/v3/role_assignments?user.id=514ea0d902e04c41a548d79008a6d68d&scope.project.id=f7edfd8f65e54a4683d5895b5809cedc&role.id=23e174ce743b46eabe618c212691da55 => generated 259 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 10:51:11.609429 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f669cc-7034-49bd-a18d-d7bde3ceb96d None None] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:51:11.609935 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f669cc-7034-49bd-a18d-d7bde3ceb96d None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:51:11.610031 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f669cc-7034-49bd-a18d-d7bde3ceb96d None None] PATH_INFO: `/` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:51:11.611602 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 49/98] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 26 10:51:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 10:51:11.616958 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba74212-8b46-40a0-973b-d81e452a0823 None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:51:11.617382 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba74212-8b46-40a0-973b-d81e452a0823 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:51:11.617556 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba74212-8b46-40a0-973b-d81e452a0823 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:51:11.640659 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-aba74212-8b46-40a0-973b-d81e452a0823 None None] MFA Rules not processed for user `ca155a48e2024c32af3fd2e19e5fac11`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 10:51:11.641869 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-aba74212-8b46-40a0-973b-d81e452a0823 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 10:51:11.652347 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a89dd0-2bd9-47d4-89f3-d2d43b68498a None None] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:51:11.652653 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a89dd0-2bd9-47d4-89f3-d2d43b68498a None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:51:11.652897 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a89dd0-2bd9-47d4-89f3-d2d43b68498a None None] PATH_INFO: `/` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:51:11.654419 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 50/99] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 26 10:51:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 26 10:51:11.663886 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e80bdf-55ef-4784-8f49-cfb220a29044 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:51:11.664176 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e80bdf-55ef-4784-8f49-cfb220a29044 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:51:11.664449 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e80bdf-55ef-4784-8f49-cfb220a29044 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:51:11.688957 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-f3e80bdf-55ef-4784-8f49-cfb220a29044 None None] MFA Rules not processed for user `ca155a48e2024c32af3fd2e19e5fac11`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 10:51:11.690784 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-f3e80bdf-55ef-4784-8f49-cfb220a29044 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 10:51:11.729642 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:51:11.772122 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 50/100] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 26 10:51:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 10:51:11.804693 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 51/101] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 26 10:51:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 10:51:12.136857 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba74212-8b46-40a0-973b-d81e452a0823 None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:51:12.236051 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e20d3ad-4f8d-45de-8f61-bdb00546e31e None None] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:51:12.236369 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e20d3ad-4f8d-45de-8f61-bdb00546e31e None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:51:12.236620 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e20d3ad-4f8d-45de-8f61-bdb00546e31e None None] PATH_INFO: `/` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:51:12.238339 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 52/102] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 26 10:51:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 10:51:12.245440 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f225dfed-dd1e-4d6b-b2ee-c2b156fa9e30 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:51:12.245723 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f225dfed-dd1e-4d6b-b2ee-c2b156fa9e30 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:51:12.246021 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f225dfed-dd1e-4d6b-b2ee-c2b156fa9e30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:51:12.279124 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-f225dfed-dd1e-4d6b-b2ee-c2b156fa9e30 None None] MFA Rules not processed for user `ca155a48e2024c32af3fd2e19e5fac11`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 10:51:12.280891 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-f225dfed-dd1e-4d6b-b2ee-c2b156fa9e30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 10:51:12.288311 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba74212-8b46-40a0-973b-d81e452a0823 None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:51:12.334992 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91963c4a-365d-41df-855f-23984c7e99a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:51:12.335642 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91963c4a-365d-41df-855f-23984c7e99a5 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:51:12.335923 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91963c4a-365d-41df-855f-23984c7e99a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:51:12.336198 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91963c4a-365d-41df-855f-23984c7e99a5 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:51:12.361501 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91963c4a-365d-41df-855f-23984c7e99a5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 10:51:12.366559 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application [None req-91963c4a-365d-41df-855f-23984c7e99a5 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 26 10:51:12.366559 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 10:51:12.366559 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 10:51:12.366559 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 10:51:12.366559 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 10:51:12.366559 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 10:51:12.366559 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 10:51:12.366559 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 10:51:12.366559 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 10:51:12.366559 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 10:51:12.366559 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 10:51:12.366559 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 10:51:12.366559 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 10:51:12.366559 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 10:51:12.366559 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 10:51:12.366559 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 10:51:12.367461 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 10:51:12.367461 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 10:51:12.367461 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 10:51:12.367461 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 10:51:12.367461 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 10:51:12.367461 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 10:51:12.367461 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 10:51:12.367461 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 10:51:12.367461 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 10:51:12.367461 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 10:51:12.367461 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 10:51:12.367461 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application with Lock( Aug 26 10:51:12.367461 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 10:51:12.367461 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._enter() Aug 26 10:51:12.367461 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 10:51:12.367461 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 10:51:12.368210 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 10:51:12.368210 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.creator() Aug 26 10:51:12.368210 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 10:51:12.368210 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application created_value = creator( Aug 26 10:51:12.368210 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 10:51:12.368210 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 10:51:12.368210 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 10:51:12.368210 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 10:51:12.368210 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 10:51:12.368210 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 10:51:12.368210 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 26 10:51:12.368210 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Aug 26 10:51:12.368210 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 51/103] 10.222.0.40 () {56 vars in 1099 bytes} [Fri Aug 26 10:51:12 2022] GET /identity/v3/roles/reader => generated 84 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 10:51:12.376750 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91963c4a-365d-41df-855f-23984c7e99a5 None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:51:12.397190 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 53/104] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 26 10:51:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 26 10:51:12.398301 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f225dfed-dd1e-4d6b-b2ee-c2b156fa9e30 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:51:12.474574 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:51:12.495445 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:51:12.599101 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91963c4a-365d-41df-855f-23984c7e99a5 None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:51:12.653805 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f225dfed-dd1e-4d6b-b2ee-c2b156fa9e30 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:51:12.667626 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a43caac-fd95-413c-b5a5-d64649a9c536 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:51:12.668282 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a43caac-fd95-413c-b5a5-d64649a9c536 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:51:12.668570 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a43caac-fd95-413c-b5a5-d64649a9c536 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:51:12.668850 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a43caac-fd95-413c-b5a5-d64649a9c536 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:51:12.673429 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a43caac-fd95-413c-b5a5-d64649a9c536 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:51:12.673429 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a43caac-fd95-413c-b5a5-d64649a9c536 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:51:12.673598 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 52/105] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Aug 26 10:51:12 2022] GET /identity/v3/domains/default => generated 199 bytes in 305 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 10:51:12.680427 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a43caac-fd95-413c-b5a5-d64649a9c536 None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:51:12.743638 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a609ff8-0af5-435b-82cc-0b31c385c2a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:51:12.744264 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a609ff8-0af5-435b-82cc-0b31c385c2a2 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:51:12.744629 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a609ff8-0af5-435b-82cc-0b31c385c2a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:51:12.744907 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a609ff8-0af5-435b-82cc-0b31c385c2a2 None admin] PATH_INFO: `/v3/roles` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:51:12.747122 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a609ff8-0af5-435b-82cc-0b31c385c2a2 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 10:51:12.747444 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a609ff8-0af5-435b-82cc-0b31c385c2a2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:51:12.748831 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a609ff8-0af5-435b-82cc-0b31c385c2a2 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:51:12.770842 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 54/106] 10.222.0.40 () {56 vars in 1101 bytes} [Fri Aug 26 10:51:12 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 373 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 10:51:12.771752 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a609ff8-0af5-435b-82cc-0b31c385c2a2 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:51:12.897859 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a43caac-fd95-413c-b5a5-d64649a9c536 None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:51:12.901735 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:51:12.905010 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a609ff8-0af5-435b-82cc-0b31c385c2a2 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:51:12.911792 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:51:12.941614 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9c539c-226a-4f8c-98f5-a752c0323f85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:51:12.942506 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9c539c-226a-4f8c-98f5-a752c0323f85 None admin] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:51:12.943205 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9c539c-226a-4f8c-98f5-a752c0323f85 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:51:12.943481 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9c539c-226a-4f8c-98f5-a752c0323f85 None admin] PATH_INFO: `/v3/projects` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:51:12.944415 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c9c539c-226a-4f8c-98f5-a752c0323f85 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:51:12.946905 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c9c539c-226a-4f8c-98f5-a752c0323f85 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:51:12.950531 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ec97ae-9a03-4276-919f-8da0e3bbd822 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:51:12.951170 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ec97ae-9a03-4276-919f-8da0e3bbd822 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:51:12.951444 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ec97ae-9a03-4276-919f-8da0e3bbd822 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:51:12.951716 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ec97ae-9a03-4276-919f-8da0e3bbd822 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:51:12.962734 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ec97ae-9a03-4276-919f-8da0e3bbd822 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 10:51:12.968088 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application [None req-78ec97ae-9a03-4276-919f-8da0e3bbd822 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 26 10:51:12.968088 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 10:51:12.968088 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 10:51:12.968088 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 10:51:12.968088 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 10:51:12.968088 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 10:51:12.968088 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 10:51:12.968088 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 10:51:12.968088 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 10:51:12.968088 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 10:51:12.968088 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 10:51:12.968088 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 10:51:12.968088 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 10:51:12.968088 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 10:51:12.968088 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 10:51:12.968088 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 10:51:12.968978 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 10:51:12.968978 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 10:51:12.968978 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 10:51:12.968978 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 10:51:12.968978 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 10:51:12.968978 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 10:51:12.968978 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 10:51:12.968978 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 10:51:12.968978 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 10:51:12.968978 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 10:51:12.968978 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 10:51:12.968978 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application with Lock( Aug 26 10:51:12.968978 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 10:51:12.968978 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._enter() Aug 26 10:51:12.968978 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 10:51:12.968978 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 10:51:12.969905 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 10:51:12.969905 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.creator() Aug 26 10:51:12.969905 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 10:51:12.969905 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application created_value = creator( Aug 26 10:51:12.969905 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 10:51:12.969905 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 10:51:12.969905 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 10:51:12.969905 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 10:51:12.969905 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 10:51:12.969905 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 10:51:12.969905 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 26 10:51:12.969905 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Aug 26 10:51:12.969905 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 55/107] 10.222.0.40 () {56 vars in 1099 bytes} [Fri Aug 26 10:51:12 2022] GET /identity/v3/roles/member => generated 84 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 10:51:12.970571 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ec97ae-9a03-4276-919f-8da0e3bbd822 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:51:12.983937 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 53/108] 10.222.0.40 () {60 vars in 1141 bytes} [Fri Aug 26 10:51:12 2022] POST /identity/v3/projects => generated 302 bytes in 304 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 26 10:51:12.985491 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de00174-bc9b-4c5a-8953-be121f68f9a6 None None] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:51:12.985942 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de00174-bc9b-4c5a-8953-be121f68f9a6 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:51:12.986399 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de00174-bc9b-4c5a-8953-be121f68f9a6 None None] PATH_INFO: `/` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:51:12.988582 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 54/109] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 26 10:51:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 10:51:12.989497 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de00174-bc9b-4c5a-8953-be121f68f9a6 None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:51:13.154257 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ec97ae-9a03-4276-919f-8da0e3bbd822 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:51:13.162311 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de00174-bc9b-4c5a-8953-be121f68f9a6 None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:51:13.201732 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd87216-d6bb-4ca9-a8a0-507e59bed631 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:51:13.202411 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd87216-d6bb-4ca9-a8a0-507e59bed631 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:51:13.202690 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd87216-d6bb-4ca9-a8a0-507e59bed631 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:51:13.202967 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd87216-d6bb-4ca9-a8a0-507e59bed631 None admin] PATH_INFO: `/v3/users/a19a4f7cc5bd49409158ab71314d727d` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:51:13.210851 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d321622e-a9a6-4883-969e-bef1786e20d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:51:13.211672 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d321622e-a9a6-4883-969e-bef1786e20d5 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:51:13.212092 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d321622e-a9a6-4883-969e-bef1786e20d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:51:13.212498 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d321622e-a9a6-4883-969e-bef1786e20d5 None admin] PATH_INFO: `/v3/roles` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:51:13.213333 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d321622e-a9a6-4883-969e-bef1786e20d5 None admin] RBAC: Adding query filter params (name=member) {{(pid=81423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 10:51:13.213861 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d321622e-a9a6-4883-969e-bef1786e20d5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:51:13.216186 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d321622e-a9a6-4883-969e-bef1786e20d5 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:51:13.223004 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd87216-d6bb-4ca9-a8a0-507e59bed631 None admin] RBAC: Authorizing `identity:get_user(user_id=a19a4f7cc5bd49409158ab71314d727d)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:51:13.224559 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd87216-d6bb-4ca9-a8a0-507e59bed631 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:51:13.226302 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 56/110] 10.222.0.40 () {56 vars in 1177 bytes} [Fri Aug 26 10:51:12 2022] GET /identity/v3/users/a19a4f7cc5bd49409158ab71314d727d => generated 289 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 10:51:13.228421 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3ae6bd-926f-43e7-a3c1-d37e5c2c1918 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:51:13.228707 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3ae6bd-926f-43e7-a3c1-d37e5c2c1918 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:51:13.228986 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3ae6bd-926f-43e7-a3c1-d37e5c2c1918 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:51:13.229578 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 55/111] 10.222.0.40 () {56 vars in 1101 bytes} [Fri Aug 26 10:51:12 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 10:51:13.230856 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7926f039-954e-4bd5-8e37-c3a35e705cbf None None] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:51:13.231338 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7926f039-954e-4bd5-8e37-c3a35e705cbf None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:51:13.231436 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7926f039-954e-4bd5-8e37-c3a35e705cbf None None] PATH_INFO: `/` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:51:13.237199 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 56/112] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 26 10:51:13 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 10:51:13.238090 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7926f039-954e-4bd5-8e37-c3a35e705cbf None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:51:13.253084 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-6e3ae6bd-926f-43e7-a3c1-d37e5c2c1918 None None] MFA Rules not processed for user `ca155a48e2024c32af3fd2e19e5fac11`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 10:51:13.254890 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-6e3ae6bd-926f-43e7-a3c1-d37e5c2c1918 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 10:51:13.264866 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:51:13.272479 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:51:13.334168 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 57/113] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 26 10:51:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 10:51:13.335301 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3ae6bd-926f-43e7-a3c1-d37e5c2c1918 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:51:13.383214 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7926f039-954e-4bd5-8e37-c3a35e705cbf None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:51:13.438994 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3ae6bd-926f-43e7-a3c1-d37e5c2c1918 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:51:13.444215 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7c4bc4-8135-4b96-a5c1-beaa150d4694 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:51:13.444979 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7c4bc4-8135-4b96-a5c1-beaa150d4694 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:51:13.445326 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7c4bc4-8135-4b96-a5c1-beaa150d4694 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:51:13.445691 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7c4bc4-8135-4b96-a5c1-beaa150d4694 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:51:13.478727 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e96898d-67fc-4373-a8d0-8b60e051780c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:51:13.479656 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e96898d-67fc-4373-a8d0-8b60e051780c None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:51:13.480143 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e96898d-67fc-4373-a8d0-8b60e051780c None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:51:13.480589 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e96898d-67fc-4373-a8d0-8b60e051780c None admin] PATH_INFO: `/v3/users/514ea0d902e04c41a548d79008a6d68d` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:51:13.482181 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e96898d-67fc-4373-a8d0-8b60e051780c None admin] RBAC: Authorizing `identity:get_user(user_id=514ea0d902e04c41a548d79008a6d68d)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:51:13.484549 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e96898d-67fc-4373-a8d0-8b60e051780c None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:51:13.486760 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 58/114] 10.222.0.40 () {56 vars in 1177 bytes} [Fri Aug 26 10:51:13 2022] GET /identity/v3/users/514ea0d902e04c41a548d79008a6d68d => generated 275 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 10:51:13.488161 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1aa30d-5c60-43a2-90ac-09f55efd584d None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:51:13.488668 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1aa30d-5c60-43a2-90ac-09f55efd584d None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:51:13.489139 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1aa30d-5c60-43a2-90ac-09f55efd584d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:51:13.507991 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-bc1aa30d-5c60-43a2-90ac-09f55efd584d None None] MFA Rules not processed for user `ca155a48e2024c32af3fd2e19e5fac11`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 10:51:13.509517 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-bc1aa30d-5c60-43a2-90ac-09f55efd584d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 10:51:13.522268 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:51:13.555690 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad7c4bc4-8135-4b96-a5c1-beaa150d4694 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:51:13.557182 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad7c4bc4-8135-4b96-a5c1-beaa150d4694 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:51:13.585891 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 59/115] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 26 10:51:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 10:51:13.587117 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1aa30d-5c60-43a2-90ac-09f55efd584d None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:51:13.594877 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:51:13.710284 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1aa30d-5c60-43a2-90ac-09f55efd584d None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:51:13.748286 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 57/116] 10.222.0.40 () {58 vars in 1282 bytes} [Fri Aug 26 10:51:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 510 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 26 10:51:13.750269 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1100c258-d1ee-4137-9009-8c5bdae54426 None None] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:51:13.750776 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1100c258-d1ee-4137-9009-8c5bdae54426 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:51:13.751298 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1100c258-d1ee-4137-9009-8c5bdae54426 None None] PATH_INFO: `/` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:51:13.753191 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 58/117] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 26 10:51:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 10:51:13.754270 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1100c258-d1ee-4137-9009-8c5bdae54426 None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:51:13.758008 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd241411-5fb1-47ea-baf5-17180958fe53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:51:13.759622 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd241411-5fb1-47ea-baf5-17180958fe53 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:51:13.759910 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd241411-5fb1-47ea-baf5-17180958fe53 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:51:13.760191 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd241411-5fb1-47ea-baf5-17180958fe53 None admin] PATH_INFO: `/v3/projects/da070dea52de4d71952ed50827d44a1c` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:51:13.762266 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd241411-5fb1-47ea-baf5-17180958fe53 None admin] RBAC: Authorizing `identity:get_project(project_id=da070dea52de4d71952ed50827d44a1c)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:51:13.763767 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd241411-5fb1-47ea-baf5-17180958fe53 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:51:13.765916 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 60/118] 10.222.0.40 () {56 vars in 1186 bytes} [Fri Aug 26 10:51:13 2022] GET /identity/v3/projects/da070dea52de4d71952ed50827d44a1c => generated 298 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 10:51:13.766720 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd241411-5fb1-47ea-baf5-17180958fe53 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:51:13.787431 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:51:13.882508 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1100c258-d1ee-4137-9009-8c5bdae54426 None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:51:13.895231 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd241411-5fb1-47ea-baf5-17180958fe53 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:51:13.911908 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:51:13.924540 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0203aeb9-843b-4828-96ba-2f3c13b58b57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:51:13.926161 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0203aeb9-843b-4828-96ba-2f3c13b58b57 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:51:13.926637 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0203aeb9-843b-4828-96ba-2f3c13b58b57 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:51:13.927100 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0203aeb9-843b-4828-96ba-2f3c13b58b57 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:51:13.934046 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd8fb01-2abc-4d92-a0f6-ff8eff56ab1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:51:13.934682 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd8fb01-2abc-4d92-a0f6-ff8eff56ab1d None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:51:13.934964 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd8fb01-2abc-4d92-a0f6-ff8eff56ab1d None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:51:13.935243 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd8fb01-2abc-4d92-a0f6-ff8eff56ab1d None admin] PATH_INFO: `/v3/projects/da070dea52de4d71952ed50827d44a1c` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:51:13.936373 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd8fb01-2abc-4d92-a0f6-ff8eff56ab1d None admin] RBAC: Authorizing `identity:get_project(project_id=da070dea52de4d71952ed50827d44a1c)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:51:13.937828 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd8fb01-2abc-4d92-a0f6-ff8eff56ab1d None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:51:13.938564 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0203aeb9-843b-4828-96ba-2f3c13b58b57 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 10:51:13.944816 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 61/119] 10.222.0.40 () {56 vars in 1186 bytes} [Fri Aug 26 10:51:13 2022] GET /identity/v3/projects/da070dea52de4d71952ed50827d44a1c => generated 298 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 10:51:13.944816 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e306b98f-330c-4b94-a62a-919122a4fbcd None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:51:13.944816 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e306b98f-330c-4b94-a62a-919122a4fbcd None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:51:13.944816 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e306b98f-330c-4b94-a62a-919122a4fbcd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:51:13.945135 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application [None req-0203aeb9-843b-4828-96ba-2f3c13b58b57 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 26 10:51:13.945135 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 10:51:13.945135 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 10:51:13.945135 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 10:51:13.945135 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 10:51:13.945135 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 10:51:13.945135 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 10:51:13.945135 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 10:51:13.945135 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 10:51:13.945135 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 10:51:13.945135 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 10:51:13.945135 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 10:51:13.945135 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 10:51:13.945135 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 10:51:13.945135 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 10:51:13.945135 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 10:51:13.945919 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 10:51:13.945919 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 10:51:13.945919 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 10:51:13.945919 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 10:51:13.945919 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 10:51:13.945919 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 10:51:13.945919 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 10:51:13.945919 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 10:51:13.945919 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 10:51:13.945919 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 10:51:13.945919 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 10:51:13.945919 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application with Lock( Aug 26 10:51:13.945919 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 10:51:13.945919 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._enter() Aug 26 10:51:13.945919 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 10:51:13.945919 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 10:51:13.946771 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 10:51:13.946771 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.creator() Aug 26 10:51:13.946771 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 10:51:13.946771 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application created_value = creator( Aug 26 10:51:13.946771 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 10:51:13.946771 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 10:51:13.946771 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 10:51:13.946771 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 10:51:13.946771 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 10:51:13.946771 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 10:51:13.946771 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 26 10:51:13.946771 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Aug 26 10:51:13.949090 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 59/120] 10.222.0.40 () {56 vars in 1096 bytes} [Fri Aug 26 10:51:13 2022] GET /identity/v3/roles/admin => generated 83 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 10:51:13.950731 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0203aeb9-843b-4828-96ba-2f3c13b58b57 None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:51:13.964113 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-e306b98f-330c-4b94-a62a-919122a4fbcd None None] MFA Rules not processed for user `ca155a48e2024c32af3fd2e19e5fac11`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 10:51:13.965266 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-e306b98f-330c-4b94-a62a-919122a4fbcd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 10:51:14.037767 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 62/121] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 26 10:51:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 10:51:14.038957 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e306b98f-330c-4b94-a62a-919122a4fbcd None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:51:14.059821 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:51:14.073306 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0203aeb9-843b-4828-96ba-2f3c13b58b57 None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:51:14.110202 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d4fa3e-d1af-4557-bbac-cd2a0f2e2b36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:51:14.111015 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d4fa3e-d1af-4557-bbac-cd2a0f2e2b36 None admin] REQUEST_METHOD: `PUT` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:51:14.111420 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d4fa3e-d1af-4557-bbac-cd2a0f2e2b36 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:51:14.111796 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d4fa3e-d1af-4557-bbac-cd2a0f2e2b36 None admin] PATH_INFO: `/v3/projects/da070dea52de4d71952ed50827d44a1c/users/514ea0d902e04c41a548d79008a6d68d/roles/23e174ce743b46eabe618c212691da55` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:51:14.114188 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d4fa3e-d1af-4557-bbac-cd2a0f2e2b36 None admin] RBAC: Authorizing `identity:create_grant(project_id=da070dea52de4d71952ed50827d44a1c, user_id=514ea0d902e04c41a548d79008a6d68d, role_id=23e174ce743b46eabe618c212691da55)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:51:14.115723 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d4fa3e-d1af-4557-bbac-cd2a0f2e2b36 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:51:14.132914 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 60/122] 10.222.0.40 () {58 vars in 1439 bytes} [Fri Aug 26 10:51:13 2022] PUT /identity/v3/projects/da070dea52de4d71952ed50827d44a1c/users/514ea0d902e04c41a548d79008a6d68d/roles/23e174ce743b46eabe618c212691da55 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 10:51:14.134256 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d4fa3e-d1af-4557-bbac-cd2a0f2e2b36 None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:51:14.163714 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e306b98f-330c-4b94-a62a-919122a4fbcd None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:51:14.195020 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:51:14.208153 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1b4cf8-26a0-47f2-b53f-c1a198bdcfcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:51:14.209156 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1b4cf8-26a0-47f2-b53f-c1a198bdcfcc None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:51:14.210559 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1b4cf8-26a0-47f2-b53f-c1a198bdcfcc None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:51:14.211093 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1b4cf8-26a0-47f2-b53f-c1a198bdcfcc None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:51:14.240298 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa1b4cf8-26a0-47f2-b53f-c1a198bdcfcc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 10:51:14.251222 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application [None req-fa1b4cf8-26a0-47f2-b53f-c1a198bdcfcc None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 26 10:51:14.251222 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 10:51:14.251222 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 10:51:14.251222 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 10:51:14.251222 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 10:51:14.251222 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 10:51:14.251222 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 10:51:14.251222 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 10:51:14.251222 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 10:51:14.251222 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 10:51:14.251222 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 10:51:14.251222 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 10:51:14.251222 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 10:51:14.251222 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 10:51:14.251222 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 10:51:14.251222 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 10:51:14.252142 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 10:51:14.252142 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 10:51:14.252142 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 10:51:14.252142 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 10:51:14.252142 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 10:51:14.252142 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 10:51:14.252142 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 10:51:14.252142 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 10:51:14.252142 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 10:51:14.252142 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 10:51:14.252142 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 10:51:14.252142 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application with Lock( Aug 26 10:51:14.252142 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 10:51:14.252142 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._enter() Aug 26 10:51:14.252142 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 10:51:14.252142 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 10:51:14.253129 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 10:51:14.253129 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.creator() Aug 26 10:51:14.253129 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 10:51:14.253129 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application created_value = creator( Aug 26 10:51:14.253129 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 10:51:14.253129 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 10:51:14.253129 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 10:51:14.253129 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 10:51:14.253129 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 10:51:14.253129 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 10:51:14.253129 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 26 10:51:14.253129 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Aug 26 10:51:14.258717 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 63/123] 10.222.0.40 () {56 vars in 1114 bytes} [Fri Aug 26 10:51:14 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 10:51:14.265467 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1b4cf8-26a0-47f2-b53f-c1a198bdcfcc None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:51:14.338229 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d4fa3e-d1af-4557-bbac-cd2a0f2e2b36 None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:51:14.382089 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:51:14.384452 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3045cd46-9e62-44d4-ad7f-ae31e6b630fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:51:14.385114 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3045cd46-9e62-44d4-ad7f-ae31e6b630fd None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:51:14.385398 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3045cd46-9e62-44d4-ad7f-ae31e6b630fd None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:51:14.385709 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3045cd46-9e62-44d4-ad7f-ae31e6b630fd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:51:14.387046 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3045cd46-9e62-44d4-ad7f-ae31e6b630fd None admin] RBAC: Adding query filter params (role.id=dae11e55a3f343ca9954bc6a1e748eed, scope.project.id=da070dea52de4d71952ed50827d44a1c, user.id=a19a4f7cc5bd49409158ab71314d727d) {{(pid=81423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 10:51:14.387046 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3045cd46-9e62-44d4-ad7f-ae31e6b630fd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:51:14.388216 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3045cd46-9e62-44d4-ad7f-ae31e6b630fd None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:51:14.397484 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 61/124] 10.222.0.40 () {56 vars in 1374 bytes} [Fri Aug 26 10:51:14 2022] GET /identity/v3/role_assignments?user.id=a19a4f7cc5bd49409158ab71314d727d&scope.project.id=da070dea52de4d71952ed50827d44a1c&role.id=dae11e55a3f343ca9954bc6a1e748eed => generated 259 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 10:51:14.398201 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3045cd46-9e62-44d4-ad7f-ae31e6b630fd None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:51:14.424305 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1b4cf8-26a0-47f2-b53f-c1a198bdcfcc None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:51:14.468267 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd42d176-2d92-451b-ba45-c5f9171a5306 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:51:14.469340 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd42d176-2d92-451b-ba45-c5f9171a5306 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:51:14.469826 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd42d176-2d92-451b-ba45-c5f9171a5306 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:51:14.470353 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd42d176-2d92-451b-ba45-c5f9171a5306 None admin] PATH_INFO: `/v3/roles` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:51:14.472649 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd42d176-2d92-451b-ba45-c5f9171a5306 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 10:51:14.473344 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd42d176-2d92-451b-ba45-c5f9171a5306 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:51:14.475033 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd42d176-2d92-451b-ba45-c5f9171a5306 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:51:14.486118 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 64/125] 10.222.0.40 () {56 vars in 1099 bytes} [Fri Aug 26 10:51:14 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 10:51:14.487755 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd42d176-2d92-451b-ba45-c5f9171a5306 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:51:14.598863 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3045cd46-9e62-44d4-ad7f-ae31e6b630fd None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:51:14.609984 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:51:14.620889 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd42d176-2d92-451b-ba45-c5f9171a5306 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:51:14.639533 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9782e27-95c0-49e4-8f2d-4cf245438aa3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:51:14.640517 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9782e27-95c0-49e4-8f2d-4cf245438aa3 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:51:14.640658 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9782e27-95c0-49e4-8f2d-4cf245438aa3 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:51:14.641003 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9782e27-95c0-49e4-8f2d-4cf245438aa3 None admin] PATH_INFO: `/v3/roles` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:51:14.641762 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9782e27-95c0-49e4-8f2d-4cf245438aa3 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 10:51:14.642212 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9782e27-95c0-49e4-8f2d-4cf245438aa3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:51:14.644306 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9782e27-95c0-49e4-8f2d-4cf245438aa3 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:51:14.664778 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f972b99-2c0f-42ca-9392-d928dd5184bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:51:14.665749 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f972b99-2c0f-42ca-9392-d928dd5184bf None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:51:14.666480 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f972b99-2c0f-42ca-9392-d928dd5184bf None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:51:14.667134 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f972b99-2c0f-42ca-9392-d928dd5184bf None admin] PATH_INFO: `/v3/roles/member` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:51:14.667497 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 62/126] 10.222.0.40 () {56 vars in 1111 bytes} [Fri Aug 26 10:51:14 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 10:51:14.668382 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9782e27-95c0-49e4-8f2d-4cf245438aa3 None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:51:14.671138 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:51:14.694199 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f972b99-2c0f-42ca-9392-d928dd5184bf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 10:51:14.698999 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application [None req-6f972b99-2c0f-42ca-9392-d928dd5184bf None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 26 10:51:14.698999 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 10:51:14.698999 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 10:51:14.698999 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 10:51:14.698999 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 10:51:14.698999 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 10:51:14.698999 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 10:51:14.698999 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 10:51:14.698999 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 10:51:14.698999 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 10:51:14.698999 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 10:51:14.698999 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 10:51:14.698999 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 10:51:14.698999 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 10:51:14.698999 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 10:51:14.698999 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 10:51:14.700051 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 10:51:14.700051 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 10:51:14.700051 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 10:51:14.700051 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 10:51:14.700051 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 10:51:14.700051 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 10:51:14.700051 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 10:51:14.700051 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 10:51:14.700051 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 10:51:14.700051 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 10:51:14.700051 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 10:51:14.700051 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application with Lock( Aug 26 10:51:14.700051 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 10:51:14.700051 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._enter() Aug 26 10:51:14.700051 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 10:51:14.700051 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 10:51:14.700993 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 10:51:14.700993 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.creator() Aug 26 10:51:14.700993 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 10:51:14.700993 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application created_value = creator( Aug 26 10:51:14.700993 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 10:51:14.700993 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 10:51:14.700993 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 10:51:14.700993 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 10:51:14.700993 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 10:51:14.700993 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 10:51:14.700993 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 26 10:51:14.700993 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Aug 26 10:51:14.700993 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 65/127] 10.222.0.40 () {56 vars in 1099 bytes} [Fri Aug 26 10:51:14 2022] GET /identity/v3/roles/member => generated 84 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 10:51:14.700993 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f972b99-2c0f-42ca-9392-d928dd5184bf None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:51:14.839253 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9782e27-95c0-49e4-8f2d-4cf245438aa3 None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:51:14.899823 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9456452-bf20-439a-8f5c-779fb54c5420 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:51:14.901050 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9456452-bf20-439a-8f5c-779fb54c5420 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:51:14.903036 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f972b99-2c0f-42ca-9392-d928dd5184bf None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:51:14.903965 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9456452-bf20-439a-8f5c-779fb54c5420 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:51:14.904439 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9456452-bf20-439a-8f5c-779fb54c5420 None admin] PATH_INFO: `/v3/users/ca155a48e2024c32af3fd2e19e5fac11` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:51:14.905857 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9456452-bf20-439a-8f5c-779fb54c5420 None admin] RBAC: Authorizing `identity:get_user(user_id=ca155a48e2024c32af3fd2e19e5fac11)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:51:14.908694 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9456452-bf20-439a-8f5c-779fb54c5420 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:51:14.916424 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 63/128] 10.222.0.40 () {56 vars in 1177 bytes} [Fri Aug 26 10:51:14 2022] GET /identity/v3/users/ca155a48e2024c32af3fd2e19e5fac11 => generated 247 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 10:51:14.917471 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9456452-bf20-439a-8f5c-779fb54c5420 None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:51:14.969165 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71cffbe5-b33e-45c0-9e07-7a1cbfc12039 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:51:14.970624 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71cffbe5-b33e-45c0-9e07-7a1cbfc12039 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:51:14.973108 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71cffbe5-b33e-45c0-9e07-7a1cbfc12039 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:51:14.973531 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71cffbe5-b33e-45c0-9e07-7a1cbfc12039 None admin] PATH_INFO: `/v3/users/514ea0d902e04c41a548d79008a6d68d` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:51:14.975347 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71cffbe5-b33e-45c0-9e07-7a1cbfc12039 None admin] RBAC: Authorizing `identity:get_user(user_id=514ea0d902e04c41a548d79008a6d68d)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:51:14.979158 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71cffbe5-b33e-45c0-9e07-7a1cbfc12039 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:51:14.984003 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 66/129] 10.222.0.40 () {56 vars in 1177 bytes} [Fri Aug 26 10:51:14 2022] GET /identity/v3/users/514ea0d902e04c41a548d79008a6d68d => generated 275 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 10:51:14.984963 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71cffbe5-b33e-45c0-9e07-7a1cbfc12039 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:51:15.000382 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:51:15.099993 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:51:15.132663 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9456452-bf20-439a-8f5c-779fb54c5420 None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:51:15.153842 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71cffbe5-b33e-45c0-9e07-7a1cbfc12039 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:51:15.176805 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216c991b-b21d-436c-a029-1f12f387d5d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:51:15.177677 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216c991b-b21d-436c-a029-1f12f387d5d6 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:51:15.177960 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216c991b-b21d-436c-a029-1f12f387d5d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:51:15.178686 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216c991b-b21d-436c-a029-1f12f387d5d6 None admin] PATH_INFO: `/v3/roles` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:51:15.179647 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-216c991b-b21d-436c-a029-1f12f387d5d6 None admin] RBAC: Adding query filter params (name=member) {{(pid=81423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 10:51:15.179956 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-216c991b-b21d-436c-a029-1f12f387d5d6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:51:15.182207 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-216c991b-b21d-436c-a029-1f12f387d5d6 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:51:15.189539 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8789a17b-9a8f-4496-a4b8-cff82a172d3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:51:15.190800 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8789a17b-9a8f-4496-a4b8-cff82a172d3d None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:51:15.191062 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8789a17b-9a8f-4496-a4b8-cff82a172d3d None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:51:15.191338 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8789a17b-9a8f-4496-a4b8-cff82a172d3d None admin] PATH_INFO: `/v3/projects/da070dea52de4d71952ed50827d44a1c` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:51:15.192481 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8789a17b-9a8f-4496-a4b8-cff82a172d3d None admin] RBAC: Authorizing `identity:get_project(project_id=da070dea52de4d71952ed50827d44a1c)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:51:15.193020 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 64/130] 10.222.0.40 () {56 vars in 1101 bytes} [Fri Aug 26 10:51:14 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 10:51:15.193862 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216c991b-b21d-436c-a029-1f12f387d5d6 None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:51:15.193994 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8789a17b-9a8f-4496-a4b8-cff82a172d3d None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:51:15.195686 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 67/131] 10.222.0.40 () {56 vars in 1186 bytes} [Fri Aug 26 10:51:14 2022] GET /identity/v3/projects/da070dea52de4d71952ed50827d44a1c => generated 298 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 10:51:15.196674 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca4eeb7-8a3c-4783-8d3d-9bedfd9118c0 None None] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:51:15.196978 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca4eeb7-8a3c-4783-8d3d-9bedfd9118c0 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:51:15.197208 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca4eeb7-8a3c-4783-8d3d-9bedfd9118c0 None None] PATH_INFO: `/` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:51:15.198613 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 68/132] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 26 10:51:15 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 10:51:15.203364 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca4eeb7-8a3c-4783-8d3d-9bedfd9118c0 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:51:15.322812 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca4eeb7-8a3c-4783-8d3d-9bedfd9118c0 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:51:15.325316 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:51:15.328616 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216c991b-b21d-436c-a029-1f12f387d5d6 None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:51:15.366463 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:51:15.368539 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4f2cd2-3ba3-400c-ad2f-bb660de00b8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:51:15.369192 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4f2cd2-3ba3-400c-ad2f-bb660de00b8a None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:51:15.369540 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4f2cd2-3ba3-400c-ad2f-bb660de00b8a None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:51:15.369954 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4f2cd2-3ba3-400c-ad2f-bb660de00b8a None admin] PATH_INFO: `/v3/projects/da070dea52de4d71952ed50827d44a1c` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:51:15.371296 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f4f2cd2-3ba3-400c-ad2f-bb660de00b8a None admin] RBAC: Authorizing `identity:get_project(project_id=da070dea52de4d71952ed50827d44a1c)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:51:15.373345 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f4f2cd2-3ba3-400c-ad2f-bb660de00b8a None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:51:15.375265 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7580f7b3-faea-4b52-a12e-66750d6312bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:51:15.375470 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 65/133] 10.222.0.40 () {56 vars in 1186 bytes} [Fri Aug 26 10:51:15 2022] GET /identity/v3/projects/da070dea52de4d71952ed50827d44a1c => generated 298 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 10:51:15.376257 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7580f7b3-faea-4b52-a12e-66750d6312bb None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:51:15.376385 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4f2cd2-3ba3-400c-ad2f-bb660de00b8a None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:51:15.376786 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7580f7b3-faea-4b52-a12e-66750d6312bb None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:51:15.377200 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7580f7b3-faea-4b52-a12e-66750d6312bb None admin] PATH_INFO: `/v3/users/514ea0d902e04c41a548d79008a6d68d` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:51:15.379264 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7580f7b3-faea-4b52-a12e-66750d6312bb None admin] RBAC: Authorizing `identity:get_user(user_id=514ea0d902e04c41a548d79008a6d68d)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:51:15.382343 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7580f7b3-faea-4b52-a12e-66750d6312bb None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:51:15.384196 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 69/134] 10.222.0.40 () {56 vars in 1177 bytes} [Fri Aug 26 10:51:15 2022] GET /identity/v3/users/514ea0d902e04c41a548d79008a6d68d => generated 275 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 10:51:15.385541 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae37d5b9-053b-4663-993a-6b2ce6021219 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:51:15.385964 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae37d5b9-053b-4663-993a-6b2ce6021219 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:51:15.386394 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae37d5b9-053b-4663-993a-6b2ce6021219 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:51:15.411838 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-ae37d5b9-053b-4663-993a-6b2ce6021219 None None] MFA Rules not processed for user `ca155a48e2024c32af3fd2e19e5fac11`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 10:51:15.413573 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-ae37d5b9-053b-4663-993a-6b2ce6021219 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 10:51:15.486622 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4f2cd2-3ba3-400c-ad2f-bb660de00b8a None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:51:15.516578 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 70/135] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 26 10:51:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 10:51:15.517696 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae37d5b9-053b-4663-993a-6b2ce6021219 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:51:15.556795 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37dd9d0-bd49-4a82-bdbc-94b02d131ace None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:51:15.558147 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37dd9d0-bd49-4a82-bdbc-94b02d131ace None admin] REQUEST_METHOD: `PUT` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:51:15.558147 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37dd9d0-bd49-4a82-bdbc-94b02d131ace None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:51:15.558147 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37dd9d0-bd49-4a82-bdbc-94b02d131ace None admin] PATH_INFO: `/v3/projects/da070dea52de4d71952ed50827d44a1c/users/ca155a48e2024c32af3fd2e19e5fac11/roles/1f97e4602e2e4d6498913a81f9ae8d6a` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:51:15.563245 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e37dd9d0-bd49-4a82-bdbc-94b02d131ace None admin] RBAC: Authorizing `identity:create_grant(project_id=da070dea52de4d71952ed50827d44a1c, user_id=ca155a48e2024c32af3fd2e19e5fac11, role_id=1f97e4602e2e4d6498913a81f9ae8d6a)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:51:15.564809 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e37dd9d0-bd49-4a82-bdbc-94b02d131ace None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:51:15.602497 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 66/136] 10.222.0.40 () {58 vars in 1439 bytes} [Fri Aug 26 10:51:15 2022] PUT /identity/v3/projects/da070dea52de4d71952ed50827d44a1c/users/ca155a48e2024c32af3fd2e19e5fac11/roles/1f97e4602e2e4d6498913a81f9ae8d6a => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 10:51:15.603470 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37dd9d0-bd49-4a82-bdbc-94b02d131ace None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:51:15.650612 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae37d5b9-053b-4663-993a-6b2ce6021219 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:51:15.667111 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:51:15.711972 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:51:15.729056 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0923e1-43d2-4621-83ec-c7c9208033a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:51:15.730090 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0923e1-43d2-4621-83ec-c7c9208033a1 None admin] REQUEST_METHOD: `PUT` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:51:15.731927 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0923e1-43d2-4621-83ec-c7c9208033a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:51:15.732364 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0923e1-43d2-4621-83ec-c7c9208033a1 None admin] PATH_INFO: `/v3/projects/da070dea52de4d71952ed50827d44a1c/users/514ea0d902e04c41a548d79008a6d68d/roles/9c439f898126444f92abb01910f1d02b` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:51:15.738050 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc0923e1-43d2-4621-83ec-c7c9208033a1 None admin] RBAC: Authorizing `identity:create_grant(project_id=da070dea52de4d71952ed50827d44a1c, user_id=514ea0d902e04c41a548d79008a6d68d, role_id=9c439f898126444f92abb01910f1d02b)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:51:15.739740 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc0923e1-43d2-4621-83ec-c7c9208033a1 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:51:15.775727 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 71/137] 10.222.0.40 () {58 vars in 1439 bytes} [Fri Aug 26 10:51:15 2022] PUT /identity/v3/projects/da070dea52de4d71952ed50827d44a1c/users/514ea0d902e04c41a548d79008a6d68d/roles/9c439f898126444f92abb01910f1d02b => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 10:51:15.793859 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37dd9d0-bd49-4a82-bdbc-94b02d131ace None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:51:15.832667 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5305d6-b56a-49ff-b935-95d69ec21981 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:51:15.833334 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5305d6-b56a-49ff-b935-95d69ec21981 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:51:15.833613 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5305d6-b56a-49ff-b935-95d69ec21981 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:51:15.833881 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5305d6-b56a-49ff-b935-95d69ec21981 None admin] PATH_INFO: `/v3/projects/f7edfd8f65e54a4683d5895b5809cedc` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:51:15.835125 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a5305d6-b56a-49ff-b935-95d69ec21981 None admin] RBAC: Authorizing `identity:get_project(project_id=f7edfd8f65e54a4683d5895b5809cedc)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:51:15.836563 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a5305d6-b56a-49ff-b935-95d69ec21981 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:51:15.838588 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 67/138] 10.222.0.40 () {56 vars in 1186 bytes} [Fri Aug 26 10:51:15 2022] GET /identity/v3/projects/f7edfd8f65e54a4683d5895b5809cedc => generated 312 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 10:51:15.844557 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0923e1-43d2-4621-83ec-c7c9208033a1 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:51:15.891338 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5305d6-b56a-49ff-b935-95d69ec21981 None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:51:15.989138 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0923e1-43d2-4621-83ec-c7c9208033a1 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:51:16.028460 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75bc5a10-83e8-44a9-8e92-8fc016bd02ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:51:16.029542 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75bc5a10-83e8-44a9-8e92-8fc016bd02ca None admin] REQUEST_METHOD: `PUT` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:51:16.030045 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75bc5a10-83e8-44a9-8e92-8fc016bd02ca None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:51:16.030530 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75bc5a10-83e8-44a9-8e92-8fc016bd02ca None admin] PATH_INFO: `/v3/projects/f7edfd8f65e54a4683d5895b5809cedc/users/514ea0d902e04c41a548d79008a6d68d/roles/23e174ce743b46eabe618c212691da55` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:51:16.032680 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75bc5a10-83e8-44a9-8e92-8fc016bd02ca None admin] RBAC: Authorizing `identity:create_grant(project_id=f7edfd8f65e54a4683d5895b5809cedc, user_id=514ea0d902e04c41a548d79008a6d68d, role_id=23e174ce743b46eabe618c212691da55)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:51:16.034331 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75bc5a10-83e8-44a9-8e92-8fc016bd02ca None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:51:16.058080 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 72/139] 10.222.0.40 () {58 vars in 1439 bytes} [Fri Aug 26 10:51:15 2022] PUT /identity/v3/projects/f7edfd8f65e54a4683d5895b5809cedc/users/514ea0d902e04c41a548d79008a6d68d/roles/23e174ce743b46eabe618c212691da55 => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 10:51:16.062247 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:51:16.070766 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5305d6-b56a-49ff-b935-95d69ec21981 None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:51:16.122732 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa8a30a-24da-4f26-9820-c62abd4629d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:51:16.123784 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa8a30a-24da-4f26-9820-c62abd4629d2 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:51:16.124290 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa8a30a-24da-4f26-9820-c62abd4629d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:51:16.124771 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa8a30a-24da-4f26-9820-c62abd4629d2 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:51:16.126488 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa8a30a-24da-4f26-9820-c62abd4629d2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:51:16.128978 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa8a30a-24da-4f26-9820-c62abd4629d2 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:51:16.130835 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 68/140] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Aug 26 10:51:15 2022] GET /identity/v3/domains/default => generated 199 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 10:51:16.136787 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75bc5a10-83e8-44a9-8e92-8fc016bd02ca None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:51:16.204600 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:51:16.267826 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75bc5a10-83e8-44a9-8e92-8fc016bd02ca None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:51:16.304410 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c426ace7-c5be-4b51-8431-a9d8ec4cd1e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:51:16.305025 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c426ace7-c5be-4b51-8431-a9d8ec4cd1e2 None admin] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:51:16.305325 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c426ace7-c5be-4b51-8431-a9d8ec4cd1e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:51:16.305584 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c426ace7-c5be-4b51-8431-a9d8ec4cd1e2 None admin] PATH_INFO: `/v3/users` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:51:16.306278 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c426ace7-c5be-4b51-8431-a9d8ec4cd1e2 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:51:16.307725 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c426ace7-c5be-4b51-8431-a9d8ec4cd1e2 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:51:16.333386 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 73/141] 10.222.0.40 () {60 vars in 1132 bytes} [Fri Aug 26 10:51:16 2022] POST /identity/v3/users => generated 283 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 10:51:16.844717 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef5b8eb-60d2-4aa5-b2c9-2a4aa5db8b3e None None] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:51:16.845416 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef5b8eb-60d2-4aa5-b2c9-2a4aa5db8b3e None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:51:16.845902 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef5b8eb-60d2-4aa5-b2c9-2a4aa5db8b3e None None] PATH_INFO: `/` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:51:16.848224 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 69/142] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 26 10:51:16 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 10:51:16.859826 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f6cf48-1612-408c-b3c1-60c3ae8e8774 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:51:16.860423 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f6cf48-1612-408c-b3c1-60c3ae8e8774 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:51:16.860903 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f6cf48-1612-408c-b3c1-60c3ae8e8774 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:51:16.888688 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-a2f6cf48-1612-408c-b3c1-60c3ae8e8774 None None] MFA Rules not processed for user `ca155a48e2024c32af3fd2e19e5fac11`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 10:51:16.889907 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-a2f6cf48-1612-408c-b3c1-60c3ae8e8774 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 10:51:16.896488 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c4a38a-fbba-423b-9528-a7a9c7e5a2a6 None None] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:51:16.896929 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c4a38a-fbba-423b-9528-a7a9c7e5a2a6 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:51:16.897108 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c4a38a-fbba-423b-9528-a7a9c7e5a2a6 None None] PATH_INFO: `/` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:51:16.899402 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 70/143] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 26 10:51:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 10:51:16.903435 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f4d0ca-4ee2-45a6-b3b5-21dbcfe7f7db None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:51:16.903727 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f4d0ca-4ee2-45a6-b3b5-21dbcfe7f7db None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:51:16.904008 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f4d0ca-4ee2-45a6-b3b5-21dbcfe7f7db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:51:16.932675 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-60f4d0ca-4ee2-45a6-b3b5-21dbcfe7f7db None None] MFA Rules not processed for user `ca155a48e2024c32af3fd2e19e5fac11`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 10:51:16.933893 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-60f4d0ca-4ee2-45a6-b3b5-21dbcfe7f7db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 10:51:17.043066 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 74/144] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 26 10:51:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 10:51:17.061101 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 71/145] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 26 10:51:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 26 10:51:17.439159 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f6cf48-1612-408c-b3c1-60c3ae8e8774 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:51:17.462647 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f4d0ca-4ee2-45a6-b3b5-21dbcfe7f7db None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:51:17.489206 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:51:17.605543 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f6cf48-1612-408c-b3c1-60c3ae8e8774 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:51:17.662406 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:51:17.664635 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c22ed3c-ce21-428e-a050-6caf8be626ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:51:17.665297 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c22ed3c-ce21-428e-a050-6caf8be626ca None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:51:17.665680 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c22ed3c-ce21-428e-a050-6caf8be626ca None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:51:17.665964 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c22ed3c-ce21-428e-a050-6caf8be626ca None admin] PATH_INFO: `/v3/roles/member` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:51:17.671400 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f4d0ca-4ee2-45a6-b3b5-21dbcfe7f7db None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:51:17.684991 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c22ed3c-ce21-428e-a050-6caf8be626ca None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 10:51:17.691009 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application [None req-6c22ed3c-ce21-428e-a050-6caf8be626ca None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 26 10:51:17.691009 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 10:51:17.691009 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 10:51:17.691009 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 10:51:17.691009 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 10:51:17.691009 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 10:51:17.691009 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 10:51:17.691009 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 10:51:17.691009 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 10:51:17.691009 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 10:51:17.691009 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 10:51:17.691009 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 10:51:17.691009 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 10:51:17.691009 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 10:51:17.691009 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 10:51:17.691009 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 10:51:17.691885 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 10:51:17.691885 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 10:51:17.691885 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 10:51:17.691885 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 10:51:17.691885 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 10:51:17.691885 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 10:51:17.691885 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 10:51:17.691885 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 10:51:17.691885 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 10:51:17.691885 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 10:51:17.691885 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 10:51:17.691885 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application with Lock( Aug 26 10:51:17.691885 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 10:51:17.691885 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._enter() Aug 26 10:51:17.691885 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 10:51:17.691885 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 10:51:17.692757 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 10:51:17.692757 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.creator() Aug 26 10:51:17.692757 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 10:51:17.692757 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application created_value = creator( Aug 26 10:51:17.692757 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 10:51:17.692757 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 10:51:17.692757 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 10:51:17.692757 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 10:51:17.692757 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 10:51:17.692757 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 10:51:17.692757 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 26 10:51:17.692757 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Aug 26 10:51:17.692757 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 75/146] 10.222.0.40 () {56 vars in 1099 bytes} [Fri Aug 26 10:51:17 2022] GET /identity/v3/roles/member => generated 84 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 10:51:17.700313 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c22ed3c-ce21-428e-a050-6caf8be626ca None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:51:17.734555 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f468b8a7-e29f-4561-a903-5488277824a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:51:17.735265 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f468b8a7-e29f-4561-a903-5488277824a1 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:51:17.735555 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f468b8a7-e29f-4561-a903-5488277824a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:51:17.735835 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f468b8a7-e29f-4561-a903-5488277824a1 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:51:17.763323 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f468b8a7-e29f-4561-a903-5488277824a1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 10:51:17.769650 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application [None req-f468b8a7-e29f-4561-a903-5488277824a1 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 26 10:51:17.769650 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 10:51:17.769650 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 10:51:17.769650 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 10:51:17.769650 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 10:51:17.769650 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 10:51:17.769650 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 10:51:17.769650 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 10:51:17.769650 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 10:51:17.769650 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 10:51:17.769650 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 10:51:17.769650 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 10:51:17.769650 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 10:51:17.769650 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 10:51:17.769650 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 10:51:17.769650 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 10:51:17.770547 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 10:51:17.770547 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 10:51:17.770547 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 10:51:17.770547 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 10:51:17.770547 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 10:51:17.770547 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 10:51:17.770547 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 10:51:17.770547 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 10:51:17.770547 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 10:51:17.770547 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 10:51:17.770547 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 10:51:17.770547 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application with Lock( Aug 26 10:51:17.770547 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 10:51:17.770547 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._enter() Aug 26 10:51:17.770547 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 10:51:17.770547 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 10:51:17.771310 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 10:51:17.771310 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.creator() Aug 26 10:51:17.771310 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 10:51:17.771310 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application created_value = creator( Aug 26 10:51:17.771310 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 10:51:17.771310 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 10:51:17.771310 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 10:51:17.771310 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 10:51:17.771310 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 10:51:17.771310 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 10:51:17.771310 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 26 10:51:17.771310 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Aug 26 10:51:17.771310 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 72/147] 10.222.0.40 () {56 vars in 1099 bytes} [Fri Aug 26 10:51:17 2022] GET /identity/v3/roles/reader => generated 84 bytes in 309 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 10:51:17.782469 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f468b8a7-e29f-4561-a903-5488277824a1 None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:51:17.851336 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:51:17.859438 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c22ed3c-ce21-428e-a050-6caf8be626ca None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:51:17.899724 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ef0859-87f9-47da-8d21-0f7b6e8a03f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:51:17.900603 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ef0859-87f9-47da-8d21-0f7b6e8a03f4 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:51:17.900987 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ef0859-87f9-47da-8d21-0f7b6e8a03f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:51:17.901364 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ef0859-87f9-47da-8d21-0f7b6e8a03f4 None admin] PATH_INFO: `/v3/roles` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:51:17.902185 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ef0859-87f9-47da-8d21-0f7b6e8a03f4 None admin] RBAC: Adding query filter params (name=member) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 10:51:17.902678 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ef0859-87f9-47da-8d21-0f7b6e8a03f4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:51:17.904137 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ef0859-87f9-47da-8d21-0f7b6e8a03f4 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:51:17.913916 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 76/148] 10.222.0.40 () {56 vars in 1101 bytes} [Fri Aug 26 10:51:17 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 10:51:17.919778 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ef0859-87f9-47da-8d21-0f7b6e8a03f4 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:51:17.989181 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f468b8a7-e29f-4561-a903-5488277824a1 None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:51:18.032914 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ef0859-87f9-47da-8d21-0f7b6e8a03f4 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:51:18.041085 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996031f4-6c39-49d9-ab7d-70b4be410dff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:51:18.041745 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996031f4-6c39-49d9-ab7d-70b4be410dff None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:51:18.042040 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996031f4-6c39-49d9-ab7d-70b4be410dff None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:51:18.042317 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996031f4-6c39-49d9-ab7d-70b4be410dff None admin] PATH_INFO: `/v3/roles` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:51:18.043001 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-996031f4-6c39-49d9-ab7d-70b4be410dff None admin] RBAC: Adding query filter params (name=reader) {{(pid=81423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 10:51:18.043304 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-996031f4-6c39-49d9-ab7d-70b4be410dff None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:51:18.044722 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-996031f4-6c39-49d9-ab7d-70b4be410dff None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:51:18.058122 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 73/149] 10.222.0.40 () {56 vars in 1101 bytes} [Fri Aug 26 10:51:17 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 10:51:18.071882 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996031f4-6c39-49d9-ab7d-70b4be410dff None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:51:18.078738 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c613aae-5ef8-4724-a72a-a8d3bbfedbbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:51:18.079386 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c613aae-5ef8-4724-a72a-a8d3bbfedbbb None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:51:18.079672 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c613aae-5ef8-4724-a72a-a8d3bbfedbbb None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:51:18.079946 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c613aae-5ef8-4724-a72a-a8d3bbfedbbb None admin] PATH_INFO: `/v3/users/514ea0d902e04c41a548d79008a6d68d` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:51:18.081701 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c613aae-5ef8-4724-a72a-a8d3bbfedbbb None admin] RBAC: Authorizing `identity:get_user(user_id=514ea0d902e04c41a548d79008a6d68d)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:51:18.083328 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c613aae-5ef8-4724-a72a-a8d3bbfedbbb None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:51:18.085120 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 77/150] 10.222.0.40 () {56 vars in 1177 bytes} [Fri Aug 26 10:51:17 2022] GET /identity/v3/users/514ea0d902e04c41a548d79008a6d68d => generated 275 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 10:51:18.090614 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c613aae-5ef8-4724-a72a-a8d3bbfedbbb None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:51:18.096054 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:51:18.120638 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:51:18.204817 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c613aae-5ef8-4724-a72a-a8d3bbfedbbb None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:51:18.225734 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996031f4-6c39-49d9-ab7d-70b4be410dff None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:51:18.262925 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fedd7dc0-09fc-4b7e-87fe-143eba9532d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:51:18.263480 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedd7dc0-09fc-4b7e-87fe-143eba9532d4 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:51:18.263813 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedd7dc0-09fc-4b7e-87fe-143eba9532d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:51:18.263978 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954b61af-62d5-4428-8de9-31b33dee5393 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:51:18.264151 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedd7dc0-09fc-4b7e-87fe-143eba9532d4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:51:18.264590 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954b61af-62d5-4428-8de9-31b33dee5393 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:51:18.264870 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954b61af-62d5-4428-8de9-31b33dee5393 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:51:18.265148 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954b61af-62d5-4428-8de9-31b33dee5393 None admin] PATH_INFO: `/v3/users/a19a4f7cc5bd49409158ab71314d727d` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:51:18.266332 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-954b61af-62d5-4428-8de9-31b33dee5393 None admin] RBAC: Authorizing `identity:get_user(user_id=a19a4f7cc5bd49409158ab71314d727d)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:51:18.267798 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-954b61af-62d5-4428-8de9-31b33dee5393 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:51:18.270990 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 74/151] 10.222.0.40 () {56 vars in 1177 bytes} [Fri Aug 26 10:51:18 2022] GET /identity/v3/users/a19a4f7cc5bd49409158ab71314d727d => generated 289 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 10:51:18.300719 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954b61af-62d5-4428-8de9-31b33dee5393 None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:51:18.347122 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fedd7dc0-09fc-4b7e-87fe-143eba9532d4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:51:18.348612 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fedd7dc0-09fc-4b7e-87fe-143eba9532d4 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:51:18.381440 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:51:18.461481 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 78/152] 10.222.0.40 () {58 vars in 1282 bytes} [Fri Aug 26 10:51:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 26 10:51:18.467935 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fedd7dc0-09fc-4b7e-87fe-143eba9532d4 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:51:18.501484 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:51:18.536112 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954b61af-62d5-4428-8de9-31b33dee5393 None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:51:18.612514 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5223f5-e6d7-4851-ab2d-8b644f404329 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:51:18.613349 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5223f5-e6d7-4851-ab2d-8b644f404329 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:51:18.613630 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5223f5-e6d7-4851-ab2d-8b644f404329 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:51:18.613906 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5223f5-e6d7-4851-ab2d-8b644f404329 None admin] PATH_INFO: `/v3/projects/da070dea52de4d71952ed50827d44a1c` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:51:18.617541 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c5223f5-e6d7-4851-ab2d-8b644f404329 None admin] RBAC: Authorizing `identity:get_project(project_id=da070dea52de4d71952ed50827d44a1c)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:51:18.619029 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c5223f5-e6d7-4851-ab2d-8b644f404329 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:51:18.620729 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 75/153] 10.222.0.40 () {56 vars in 1186 bytes} [Fri Aug 26 10:51:18 2022] GET /identity/v3/projects/da070dea52de4d71952ed50827d44a1c => generated 298 bytes in 320 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 10:51:18.641244 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fedd7dc0-09fc-4b7e-87fe-143eba9532d4 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:51:18.647750 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5223f5-e6d7-4851-ab2d-8b644f404329 None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:51:18.700062 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:51:18.725378 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3636599-c1bf-4341-84df-390af5d1bccc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:51:18.726045 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3636599-c1bf-4341-84df-390af5d1bccc None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:51:18.726328 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3636599-c1bf-4341-84df-390af5d1bccc None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:51:18.726604 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3636599-c1bf-4341-84df-390af5d1bccc None admin] PATH_INFO: `/v3/projects/da070dea52de4d71952ed50827d44a1c` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:51:18.729165 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3636599-c1bf-4341-84df-390af5d1bccc None admin] RBAC: Authorizing `identity:get_project(project_id=da070dea52de4d71952ed50827d44a1c)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:51:18.730770 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3636599-c1bf-4341-84df-390af5d1bccc None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:51:18.739503 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 79/154] 10.222.0.40 () {56 vars in 1186 bytes} [Fri Aug 26 10:51:18 2022] GET /identity/v3/projects/da070dea52de4d71952ed50827d44a1c => generated 298 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 10:51:18.758443 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3636599-c1bf-4341-84df-390af5d1bccc None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:51:18.857016 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5223f5-e6d7-4851-ab2d-8b644f404329 None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:51:18.892287 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3636599-c1bf-4341-84df-390af5d1bccc None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:51:18.930452 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2100fb-46bf-4567-b47c-caa3967680fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:51:18.931616 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2100fb-46bf-4567-b47c-caa3967680fd None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:51:18.932054 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2100fb-46bf-4567-b47c-caa3967680fd None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:51:18.932475 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2100fb-46bf-4567-b47c-caa3967680fd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:51:18.933366 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc2100fb-46bf-4567-b47c-caa3967680fd None admin] RBAC: Adding query filter params (role.id=23e174ce743b46eabe618c212691da55, scope.project.id=da070dea52de4d71952ed50827d44a1c, user.id=514ea0d902e04c41a548d79008a6d68d) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 10:51:18.933842 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc2100fb-46bf-4567-b47c-caa3967680fd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:51:18.935613 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc2100fb-46bf-4567-b47c-caa3967680fd None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:51:18.942778 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 80/155] 10.222.0.40 () {56 vars in 1374 bytes} [Fri Aug 26 10:51:18 2022] GET /identity/v3/role_assignments?user.id=514ea0d902e04c41a548d79008a6d68d&scope.project.id=da070dea52de4d71952ed50827d44a1c&role.id=23e174ce743b46eabe618c212691da55 => generated 608 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 10:51:18.944330 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7631b10b-a5d7-4a40-a6ed-aa572adf408b None None] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:51:18.944859 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7631b10b-a5d7-4a40-a6ed-aa572adf408b None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:51:18.945369 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7631b10b-a5d7-4a40-a6ed-aa572adf408b None None] PATH_INFO: `/` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:51:18.946803 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 81/156] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 26 10:51:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 10:51:18.947652 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbee430e-809e-49ab-a300-a6f98cdfb3b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:51:18.948463 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbee430e-809e-49ab-a300-a6f98cdfb3b2 None admin] REQUEST_METHOD: `PUT` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:51:18.948816 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbee430e-809e-49ab-a300-a6f98cdfb3b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:51:18.949104 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbee430e-809e-49ab-a300-a6f98cdfb3b2 None admin] PATH_INFO: `/v3/projects/da070dea52de4d71952ed50827d44a1c/users/a19a4f7cc5bd49409158ab71314d727d/roles/dae11e55a3f343ca9954bc6a1e748eed` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:51:18.951224 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbee430e-809e-49ab-a300-a6f98cdfb3b2 None admin] RBAC: Authorizing `identity:create_grant(project_id=da070dea52de4d71952ed50827d44a1c, user_id=a19a4f7cc5bd49409158ab71314d727d, role_id=dae11e55a3f343ca9954bc6a1e748eed)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:51:18.952873 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbee430e-809e-49ab-a300-a6f98cdfb3b2 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:51:18.954379 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f09cf15-487a-4eb3-9b3f-cdb42ac78147 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:51:18.954857 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f09cf15-487a-4eb3-9b3f-cdb42ac78147 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:51:18.955311 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f09cf15-487a-4eb3-9b3f-cdb42ac78147 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:51:18.980205 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-6f09cf15-487a-4eb3-9b3f-cdb42ac78147 None None] MFA Rules not processed for user `ca155a48e2024c32af3fd2e19e5fac11`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 10:51:18.981830 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-6f09cf15-487a-4eb3-9b3f-cdb42ac78147 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 10:51:18.995622 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 76/157] 10.222.0.40 () {58 vars in 1439 bytes} [Fri Aug 26 10:51:18 2022] PUT /identity/v3/projects/da070dea52de4d71952ed50827d44a1c/users/a19a4f7cc5bd49409158ab71314d727d/roles/dae11e55a3f343ca9954bc6a1e748eed => generated 0 bytes in 348 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 26 10:51:19.055400 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:51:19.075255 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 82/158] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 26 10:51:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 10:51:19.457383 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb46a442-3c8e-477b-a798-57e4a7d13aaa None None] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:51:19.457670 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb46a442-3c8e-477b-a798-57e4a7d13aaa None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:51:19.457952 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb46a442-3c8e-477b-a798-57e4a7d13aaa None None] PATH_INFO: `/` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:51:19.459739 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 77/159] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 26 10:51:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 10:51:19.465328 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0c4d17-fd40-4954-b2f1-1d597edfa964 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:51:19.465775 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0c4d17-fd40-4954-b2f1-1d597edfa964 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:51:19.465936 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0c4d17-fd40-4954-b2f1-1d597edfa964 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:51:19.475953 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59bc0f0-8df8-4d9e-bf42-887a7f2a8e2f None None] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:51:19.476264 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59bc0f0-8df8-4d9e-bf42-887a7f2a8e2f None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:51:19.476501 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59bc0f0-8df8-4d9e-bf42-887a7f2a8e2f None None] PATH_INFO: `/` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:51:19.478083 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 78/160] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 26 10:51:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 10:51:19.484446 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0313676e-9a65-470e-85af-f9e0b9808c61 None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:51:19.484725 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0313676e-9a65-470e-85af-f9e0b9808c61 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:51:19.485001 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0313676e-9a65-470e-85af-f9e0b9808c61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:51:19.486590 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-0d0c4d17-fd40-4954-b2f1-1d597edfa964 None None] MFA Rules not processed for user `ca155a48e2024c32af3fd2e19e5fac11`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 10:51:19.488215 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-0d0c4d17-fd40-4954-b2f1-1d597edfa964 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 10:51:19.505747 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-0313676e-9a65-470e-85af-f9e0b9808c61 None None] MFA Rules not processed for user `ca155a48e2024c32af3fd2e19e5fac11`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 10:51:19.507000 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-0313676e-9a65-470e-85af-f9e0b9808c61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 10:51:19.522324 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:51:19.615475 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 83/161] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 26 10:51:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 26 10:51:19.627131 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 79/162] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 26 10:51:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 26 10:51:19.673964 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0c4d17-fd40-4954-b2f1-1d597edfa964 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:51:19.807457 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0c4d17-fd40-4954-b2f1-1d597edfa964 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:51:19.834005 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:51:19.862007 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b7ebdd-e0e1-4361-8da3-7f118e6c36fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:51:19.863271 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b7ebdd-e0e1-4361-8da3-7f118e6c36fe None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:51:19.863766 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b7ebdd-e0e1-4361-8da3-7f118e6c36fe None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:51:19.864261 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b7ebdd-e0e1-4361-8da3-7f118e6c36fe None admin] PATH_INFO: `/v3/roles/member` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:51:19.887483 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12b7ebdd-e0e1-4361-8da3-7f118e6c36fe None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 10:51:19.892640 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application [None req-12b7ebdd-e0e1-4361-8da3-7f118e6c36fe None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 26 10:51:19.892640 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 10:51:19.892640 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 10:51:19.892640 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 10:51:19.892640 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 10:51:19.892640 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 10:51:19.892640 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 10:51:19.892640 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 10:51:19.892640 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 10:51:19.892640 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 10:51:19.892640 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 10:51:19.892640 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 10:51:19.892640 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 10:51:19.892640 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 10:51:19.892640 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 10:51:19.892640 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 10:51:19.894169 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 10:51:19.894169 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 10:51:19.894169 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 10:51:19.894169 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 10:51:19.894169 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 10:51:19.894169 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 10:51:19.894169 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 10:51:19.894169 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 10:51:19.894169 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 10:51:19.894169 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 10:51:19.894169 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 10:51:19.894169 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application with Lock( Aug 26 10:51:19.894169 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 10:51:19.894169 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._enter() Aug 26 10:51:19.894169 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 10:51:19.894169 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 10:51:19.895110 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 10:51:19.895110 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.creator() Aug 26 10:51:19.895110 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 10:51:19.895110 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application created_value = creator( Aug 26 10:51:19.895110 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 10:51:19.895110 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 10:51:19.895110 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 10:51:19.895110 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 10:51:19.895110 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 10:51:19.895110 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 10:51:19.895110 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 26 10:51:19.895110 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Aug 26 10:51:19.897449 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 84/163] 10.222.0.40 () {56 vars in 1099 bytes} [Fri Aug 26 10:51:19 2022] GET /identity/v3/roles/member => generated 84 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 10:51:19.903271 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0313676e-9a65-470e-85af-f9e0b9808c61 None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:51:19.957474 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b7ebdd-e0e1-4361-8da3-7f118e6c36fe None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:51:20.031402 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0313676e-9a65-470e-85af-f9e0b9808c61 None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:51:20.076435 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e8cf89-204e-4bfa-8d0d-3d5cea567438 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:51:20.076981 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e8cf89-204e-4bfa-8d0d-3d5cea567438 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:51:20.077261 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e8cf89-204e-4bfa-8d0d-3d5cea567438 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:51:20.077598 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e8cf89-204e-4bfa-8d0d-3d5cea567438 None admin] PATH_INFO: `/v3/roles` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:51:20.078311 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56e8cf89-204e-4bfa-8d0d-3d5cea567438 None admin] RBAC: Adding query filter params (name=member) {{(pid=81423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 10:51:20.078613 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56e8cf89-204e-4bfa-8d0d-3d5cea567438 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:51:20.080187 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56e8cf89-204e-4bfa-8d0d-3d5cea567438 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:51:20.089730 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 80/164] 10.222.0.40 () {56 vars in 1101 bytes} [Fri Aug 26 10:51:19 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 10:51:20.095252 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e8cf89-204e-4bfa-8d0d-3d5cea567438 None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:51:20.113885 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:51:20.168368 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b7ebdd-e0e1-4361-8da3-7f118e6c36fe None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:51:20.233730 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1716b2-2491-4ed6-8f06-4f1c453eead2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:51:20.234408 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1716b2-2491-4ed6-8f06-4f1c453eead2 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:51:20.234687 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1716b2-2491-4ed6-8f06-4f1c453eead2 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:51:20.234962 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1716b2-2491-4ed6-8f06-4f1c453eead2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:51:20.239830 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e8cf89-204e-4bfa-8d0d-3d5cea567438 None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:51:20.258739 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d1716b2-2491-4ed6-8f06-4f1c453eead2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 10:51:20.263799 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application [None req-9d1716b2-2491-4ed6-8f06-4f1c453eead2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 26 10:51:20.263799 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 10:51:20.263799 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 10:51:20.263799 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 10:51:20.263799 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 10:51:20.263799 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 10:51:20.263799 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 10:51:20.263799 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 10:51:20.263799 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 10:51:20.263799 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 10:51:20.263799 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 10:51:20.263799 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 10:51:20.263799 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 10:51:20.263799 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 10:51:20.263799 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 10:51:20.263799 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 10:51:20.264761 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 10:51:20.264761 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 10:51:20.264761 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 10:51:20.264761 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 10:51:20.264761 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 10:51:20.264761 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 10:51:20.264761 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 10:51:20.264761 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 10:51:20.264761 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 10:51:20.264761 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 10:51:20.264761 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 10:51:20.264761 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application with Lock( Aug 26 10:51:20.264761 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 10:51:20.264761 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._enter() Aug 26 10:51:20.264761 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 10:51:20.264761 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 10:51:20.265516 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 10:51:20.265516 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.creator() Aug 26 10:51:20.265516 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 10:51:20.265516 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application created_value = creator( Aug 26 10:51:20.265516 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 10:51:20.265516 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 10:51:20.265516 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 10:51:20.265516 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 10:51:20.265516 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 10:51:20.265516 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 10:51:20.265516 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 26 10:51:20.265516 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Aug 26 10:51:20.266465 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 85/165] 10.222.0.40 () {56 vars in 1096 bytes} [Fri Aug 26 10:51:19 2022] GET /identity/v3/roles/admin => generated 83 bytes in 309 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 10:51:20.267255 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1716b2-2491-4ed6-8f06-4f1c453eead2 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:51:20.320631 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f932fbb-9645-48f2-93f2-44a979384399 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:51:20.321277 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f932fbb-9645-48f2-93f2-44a979384399 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:51:20.321555 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f932fbb-9645-48f2-93f2-44a979384399 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:51:20.321825 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f932fbb-9645-48f2-93f2-44a979384399 None admin] PATH_INFO: `/v3/users/514ea0d902e04c41a548d79008a6d68d` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:51:20.323953 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f932fbb-9645-48f2-93f2-44a979384399 None admin] RBAC: Authorizing `identity:get_user(user_id=514ea0d902e04c41a548d79008a6d68d)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:51:20.325403 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f932fbb-9645-48f2-93f2-44a979384399 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:51:20.327098 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 81/166] 10.222.0.40 () {56 vars in 1177 bytes} [Fri Aug 26 10:51:20 2022] GET /identity/v3/users/514ea0d902e04c41a548d79008a6d68d => generated 275 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 10:51:20.327805 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f932fbb-9645-48f2-93f2-44a979384399 None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:51:20.366345 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:51:20.506343 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:51:20.508134 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1716b2-2491-4ed6-8f06-4f1c453eead2 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:51:20.555379 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f932fbb-9645-48f2-93f2-44a979384399 None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:51:20.565758 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7a457b-cc47-4444-96c7-9250944dc4df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:51:20.566537 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7a457b-cc47-4444-96c7-9250944dc4df None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:51:20.566880 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7a457b-cc47-4444-96c7-9250944dc4df None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:51:20.567222 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7a457b-cc47-4444-96c7-9250944dc4df None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:51:20.584219 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f7a457b-cc47-4444-96c7-9250944dc4df None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 10:51:20.593108 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application [None req-8f7a457b-cc47-4444-96c7-9250944dc4df None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 26 10:51:20.593108 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 10:51:20.593108 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 10:51:20.593108 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 10:51:20.593108 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 10:51:20.593108 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 10:51:20.593108 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 10:51:20.593108 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 10:51:20.593108 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 10:51:20.593108 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 10:51:20.593108 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 10:51:20.593108 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 10:51:20.593108 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 10:51:20.593108 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 10:51:20.593108 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 10:51:20.593108 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 10:51:20.594016 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 10:51:20.594016 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 10:51:20.594016 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 10:51:20.594016 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 10:51:20.594016 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 10:51:20.594016 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 10:51:20.594016 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 10:51:20.594016 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 10:51:20.594016 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 10:51:20.594016 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 10:51:20.594016 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 10:51:20.594016 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application with Lock( Aug 26 10:51:20.594016 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 10:51:20.594016 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._enter() Aug 26 10:51:20.594016 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 10:51:20.594016 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 10:51:20.594885 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 10:51:20.594885 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.creator() Aug 26 10:51:20.594885 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 10:51:20.594885 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application created_value = creator( Aug 26 10:51:20.594885 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 10:51:20.594885 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 10:51:20.594885 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 10:51:20.594885 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 10:51:20.594885 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 10:51:20.594885 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 10:51:20.594885 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 26 10:51:20.594885 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Aug 26 10:51:20.594885 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 86/167] 10.222.0.40 () {56 vars in 1114 bytes} [Fri Aug 26 10:51:20 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 327 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 10:51:20.595551 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7a457b-cc47-4444-96c7-9250944dc4df None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:51:20.626682 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361b11ff-cfcc-445d-a55d-dce9b31e9ee2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:51:20.627310 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361b11ff-cfcc-445d-a55d-dce9b31e9ee2 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:51:20.627656 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361b11ff-cfcc-445d-a55d-dce9b31e9ee2 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:51:20.628039 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361b11ff-cfcc-445d-a55d-dce9b31e9ee2 None admin] PATH_INFO: `/v3/roles` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:51:20.628731 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-361b11ff-cfcc-445d-a55d-dce9b31e9ee2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 10:51:20.629040 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-361b11ff-cfcc-445d-a55d-dce9b31e9ee2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:51:20.630446 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-361b11ff-cfcc-445d-a55d-dce9b31e9ee2 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:51:20.648166 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 82/168] 10.222.0.40 () {56 vars in 1099 bytes} [Fri Aug 26 10:51:20 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 321 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 10:51:20.649017 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361b11ff-cfcc-445d-a55d-dce9b31e9ee2 None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:51:20.745900 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:51:20.756042 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7a457b-cc47-4444-96c7-9250944dc4df None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:51:20.825576 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361b11ff-cfcc-445d-a55d-dce9b31e9ee2 None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:51:20.859167 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9161a6ba-8c85-4941-a35d-0104bef9c97e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:51:20.859729 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9161a6ba-8c85-4941-a35d-0104bef9c97e None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:51:20.860066 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9161a6ba-8c85-4941-a35d-0104bef9c97e None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:51:20.860274 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9161a6ba-8c85-4941-a35d-0104bef9c97e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:51:20.863258 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5078192d-48b9-4648-a656-b6b033b5fd22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:51:20.863866 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5078192d-48b9-4648-a656-b6b033b5fd22 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:51:20.864139 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5078192d-48b9-4648-a656-b6b033b5fd22 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:51:20.864411 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5078192d-48b9-4648-a656-b6b033b5fd22 None admin] PATH_INFO: `/v3/roles` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:51:20.865074 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5078192d-48b9-4648-a656-b6b033b5fd22 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 10:51:20.865371 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5078192d-48b9-4648-a656-b6b033b5fd22 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:51:20.866795 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5078192d-48b9-4648-a656-b6b033b5fd22 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:51:20.876630 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 83/169] 10.222.0.40 () {56 vars in 1111 bytes} [Fri Aug 26 10:51:20 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 10:51:20.877090 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:51:20.884270 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5078192d-48b9-4648-a656-b6b033b5fd22 None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:51:21.024592 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9161a6ba-8c85-4941-a35d-0104bef9c97e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:51:21.026574 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9161a6ba-8c85-4941-a35d-0104bef9c97e None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:51:21.139865 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:51:21.144031 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 87/170] 10.222.0.40 () {58 vars in 1282 bytes} [Fri Aug 26 10:51:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 549 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 26 10:51:21.145519 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9161a6ba-8c85-4941-a35d-0104bef9c97e None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:51:21.186735 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5078192d-48b9-4648-a656-b6b033b5fd22 None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:51:21.254930 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8525349-c02e-4106-9433-c520d1fdb2f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:51:21.255581 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8525349-c02e-4106-9433-c520d1fdb2f7 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:51:21.255858 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8525349-c02e-4106-9433-c520d1fdb2f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:51:21.256130 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8525349-c02e-4106-9433-c520d1fdb2f7 None admin] PATH_INFO: `/v3/users/ca155a48e2024c32af3fd2e19e5fac11` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:51:21.257390 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8525349-c02e-4106-9433-c520d1fdb2f7 None admin] RBAC: Authorizing `identity:get_user(user_id=ca155a48e2024c32af3fd2e19e5fac11)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:51:21.258987 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8525349-c02e-4106-9433-c520d1fdb2f7 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:51:21.260583 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 84/171] 10.222.0.40 () {56 vars in 1177 bytes} [Fri Aug 26 10:51:20 2022] GET /identity/v3/users/ca155a48e2024c32af3fd2e19e5fac11 => generated 247 bytes in 383 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 10:51:21.266194 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9161a6ba-8c85-4941-a35d-0104bef9c97e None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:51:21.270004 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c16f88-22c5-47c3-81bf-af0f5f4e4bff None None] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:51:21.270555 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c16f88-22c5-47c3-81bf-af0f5f4e4bff None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:51:21.271032 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c16f88-22c5-47c3-81bf-af0f5f4e4bff None None] PATH_INFO: `/` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:51:21.272619 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 85/172] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 26 10:51:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 10:51:21.273520 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c16f88-22c5-47c3-81bf-af0f5f4e4bff None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:51:21.305385 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df5e530-f7d2-45a7-b447-f2f1fc9ffbcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:51:21.306364 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df5e530-f7d2-45a7-b447-f2f1fc9ffbcb None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:51:21.307026 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df5e530-f7d2-45a7-b447-f2f1fc9ffbcb None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:51:21.307440 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df5e530-f7d2-45a7-b447-f2f1fc9ffbcb None admin] PATH_INFO: `/v3/users/514ea0d902e04c41a548d79008a6d68d` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:51:21.308868 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df5e530-f7d2-45a7-b447-f2f1fc9ffbcb None admin] RBAC: Authorizing `identity:get_user(user_id=514ea0d902e04c41a548d79008a6d68d)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:51:21.310790 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df5e530-f7d2-45a7-b447-f2f1fc9ffbcb None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:51:21.312546 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 88/173] 10.222.0.40 () {56 vars in 1177 bytes} [Fri Aug 26 10:51:21 2022] GET /identity/v3/users/514ea0d902e04c41a548d79008a6d68d => generated 275 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 10:51:21.313614 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df5e530-f7d2-45a7-b447-f2f1fc9ffbcb None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:51:21.392593 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:51:21.413760 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:51:21.438870 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c16f88-22c5-47c3-81bf-af0f5f4e4bff None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:51:21.462225 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df5e530-f7d2-45a7-b447-f2f1fc9ffbcb None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:51:21.476764 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cdfe17d-8725-4d51-b5f2-085589baa97c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:51:21.477449 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cdfe17d-8725-4d51-b5f2-085589baa97c None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:51:21.477729 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cdfe17d-8725-4d51-b5f2-085589baa97c None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:51:21.478020 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cdfe17d-8725-4d51-b5f2-085589baa97c None admin] PATH_INFO: `/v3/projects/f7edfd8f65e54a4683d5895b5809cedc` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:51:21.479192 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cdfe17d-8725-4d51-b5f2-085589baa97c None admin] RBAC: Authorizing `identity:get_project(project_id=f7edfd8f65e54a4683d5895b5809cedc)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:51:21.480684 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cdfe17d-8725-4d51-b5f2-085589baa97c None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:51:21.482422 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 86/174] 10.222.0.40 () {56 vars in 1186 bytes} [Fri Aug 26 10:51:21 2022] GET /identity/v3/projects/f7edfd8f65e54a4683d5895b5809cedc => generated 312 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 10:51:21.483769 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04cde3a0-6ef8-4273-a53c-c09b5f01ffa1 None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:51:21.484104 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04cde3a0-6ef8-4273-a53c-c09b5f01ffa1 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:51:21.484382 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04cde3a0-6ef8-4273-a53c-c09b5f01ffa1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:51:21.502833 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91c67ef-9de7-4eea-b1d0-a4329f634e3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:51:21.503044 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-04cde3a0-6ef8-4273-a53c-c09b5f01ffa1 None None] MFA Rules not processed for user `ca155a48e2024c32af3fd2e19e5fac11`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 10:51:21.503525 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91c67ef-9de7-4eea-b1d0-a4329f634e3c None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:51:21.503806 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91c67ef-9de7-4eea-b1d0-a4329f634e3c None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:51:21.504137 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91c67ef-9de7-4eea-b1d0-a4329f634e3c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:51:21.504231 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-04cde3a0-6ef8-4273-a53c-c09b5f01ffa1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 10:51:21.578015 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 87/175] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 26 10:51:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 10:51:21.578015 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04cde3a0-6ef8-4273-a53c-c09b5f01ffa1 None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:51:21.591399 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e91c67ef-9de7-4eea-b1d0-a4329f634e3c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:51:21.592957 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e91c67ef-9de7-4eea-b1d0-a4329f634e3c None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:51:21.648996 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:51:21.674752 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:51:21.695289 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04cde3a0-6ef8-4273-a53c-c09b5f01ffa1 None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:51:21.731111 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 89/176] 10.222.0.40 () {58 vars in 1282 bytes} [Fri Aug 26 10:51:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 418 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 26 10:51:21.732526 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8ad81c-31bb-4cc7-a9f3-212a29932683 None None] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:51:21.733165 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8ad81c-31bb-4cc7-a9f3-212a29932683 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:51:21.733531 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8ad81c-31bb-4cc7-a9f3-212a29932683 None None] PATH_INFO: `/` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:51:21.734901 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 90/177] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 26 10:51:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 10:51:21.739138 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f7ffb0-2d33-41d0-b283-06e1ea733516 None None] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:51:21.739636 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f7ffb0-2d33-41d0-b283-06e1ea733516 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:51:21.739919 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f7ffb0-2d33-41d0-b283-06e1ea733516 None None] PATH_INFO: `/` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:51:21.741667 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 91/178] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 26 10:51:21 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 10:51:21.743581 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729f3dbe-138c-41e7-a7b4-126e2b7cfb5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:51:21.744201 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729f3dbe-138c-41e7-a7b4-126e2b7cfb5d None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:51:21.744543 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729f3dbe-138c-41e7-a7b4-126e2b7cfb5d None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:51:21.744818 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729f3dbe-138c-41e7-a7b4-126e2b7cfb5d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:51:21.745568 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f7ffb0-2d33-41d0-b283-06e1ea733516 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:51:21.816388 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-729f3dbe-138c-41e7-a7b4-126e2b7cfb5d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:51:21.818129 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-729f3dbe-138c-41e7-a7b4-126e2b7cfb5d None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:51:21.857969 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f7ffb0-2d33-41d0-b283-06e1ea733516 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:51:21.883135 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:51:21.898266 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afea2a14-136c-42d2-b93e-f32eeb97fa3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:51:21.898878 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afea2a14-136c-42d2-b93e-f32eeb97fa3e None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:51:21.899208 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afea2a14-136c-42d2-b93e-f32eeb97fa3e None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:51:21.899428 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afea2a14-136c-42d2-b93e-f32eeb97fa3e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:51:21.900322 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afea2a14-136c-42d2-b93e-f32eeb97fa3e None admin] RBAC: Adding query filter params (role.id=23e174ce743b46eabe618c212691da55, scope.project.id=f7edfd8f65e54a4683d5895b5809cedc, user.id=514ea0d902e04c41a548d79008a6d68d) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 10:51:21.900447 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afea2a14-136c-42d2-b93e-f32eeb97fa3e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:51:21.901884 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afea2a14-136c-42d2-b93e-f32eeb97fa3e None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:51:21.907567 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 92/179] 10.222.0.40 () {56 vars in 1374 bytes} [Fri Aug 26 10:51:21 2022] GET /identity/v3/role_assignments?user.id=514ea0d902e04c41a548d79008a6d68d&scope.project.id=f7edfd8f65e54a4683d5895b5809cedc&role.id=23e174ce743b46eabe618c212691da55 => generated 608 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 10:51:21.908756 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4134840e-e9ae-42c5-bd44-f329ec703706 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:51:21.909227 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4134840e-e9ae-42c5-bd44-f329ec703706 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:51:21.909609 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4134840e-e9ae-42c5-bd44-f329ec703706 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:51:21.926181 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 88/180] 10.222.0.40 () {58 vars in 1282 bytes} [Fri Aug 26 10:51:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 26 10:51:21.926181 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729f3dbe-138c-41e7-a7b4-126e2b7cfb5d None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:51:21.931755 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-4134840e-e9ae-42c5-bd44-f329ec703706 None None] MFA Rules not processed for user `ca155a48e2024c32af3fd2e19e5fac11`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 10:51:21.933401 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-4134840e-e9ae-42c5-bd44-f329ec703706 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 10:51:21.960047 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:51:22.010381 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 93/181] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 26 10:51:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 10:51:22.011845 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da2860c-a497-4eac-86bf-ecb46178e918 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:51:22.012203 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da2860c-a497-4eac-86bf-ecb46178e918 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:51:22.012641 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da2860c-a497-4eac-86bf-ecb46178e918 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:51:22.032780 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-0da2860c-a497-4eac-86bf-ecb46178e918 None None] MFA Rules not processed for user `ca155a48e2024c32af3fd2e19e5fac11`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 10:51:22.034219 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-0da2860c-a497-4eac-86bf-ecb46178e918 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 10:51:22.085019 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729f3dbe-138c-41e7-a7b4-126e2b7cfb5d None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:51:22.120316 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 94/182] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 26 10:51:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 10:51:22.121850 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebf5ab1-26ef-4b5f-a289-7110afad7cac None None] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:51:22.122388 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebf5ab1-26ef-4b5f-a289-7110afad7cac None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:51:22.122871 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebf5ab1-26ef-4b5f-a289-7110afad7cac None None] PATH_INFO: `/` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:51:22.124299 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 95/183] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 26 10:51:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 10:51:22.125153 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ebf5ab1-26ef-4b5f-a289-7110afad7cac None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:51:22.142206 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74ea68d-f0c5-497a-89ca-57cfe6d8dcf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:51:22.143092 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74ea68d-f0c5-497a-89ca-57cfe6d8dcf4 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:51:22.146015 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74ea68d-f0c5-497a-89ca-57cfe6d8dcf4 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:51:22.146015 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74ea68d-f0c5-497a-89ca-57cfe6d8dcf4 None admin] PATH_INFO: `/v3/projects/da070dea52de4d71952ed50827d44a1c` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:51:22.148438 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f74ea68d-f0c5-497a-89ca-57cfe6d8dcf4 None admin] RBAC: Authorizing `identity:get_project(project_id=da070dea52de4d71952ed50827d44a1c)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:51:22.149951 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f74ea68d-f0c5-497a-89ca-57cfe6d8dcf4 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:51:22.151965 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 89/184] 10.222.0.40 () {56 vars in 1186 bytes} [Fri Aug 26 10:51:21 2022] GET /identity/v3/projects/da070dea52de4d71952ed50827d44a1c => generated 298 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 10:51:22.153887 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74ea68d-f0c5-497a-89ca-57cfe6d8dcf4 None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:51:22.191338 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:51:22.250940 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:51:22.267950 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ebf5ab1-26ef-4b5f-a289-7110afad7cac None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:51:22.270316 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74ea68d-f0c5-497a-89ca-57cfe6d8dcf4 None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:51:22.307201 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2a8b44-5c84-403f-9756-1b36d1480db2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:51:22.307518 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ee4bfe-123c-4961-b29a-9a21b57c0af2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:51:22.307843 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2a8b44-5c84-403f-9756-1b36d1480db2 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:51:22.308124 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2a8b44-5c84-403f-9756-1b36d1480db2 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:51:22.308215 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ee4bfe-123c-4961-b29a-9a21b57c0af2 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:51:22.308402 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2a8b44-5c84-403f-9756-1b36d1480db2 None admin] PATH_INFO: `/v3/projects/da070dea52de4d71952ed50827d44a1c` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:51:22.308494 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ee4bfe-123c-4961-b29a-9a21b57c0af2 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:51:22.308768 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ee4bfe-123c-4961-b29a-9a21b57c0af2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:51:22.309537 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a2a8b44-5c84-403f-9756-1b36d1480db2 None admin] RBAC: Authorizing `identity:get_project(project_id=da070dea52de4d71952ed50827d44a1c)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:51:22.312174 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a2a8b44-5c84-403f-9756-1b36d1480db2 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:51:22.312979 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 90/185] 10.222.0.40 () {56 vars in 1186 bytes} [Fri Aug 26 10:51:22 2022] GET /identity/v3/projects/da070dea52de4d71952ed50827d44a1c => generated 298 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 10:51:22.314228 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2d0b51-c83f-4531-994a-a0d42106a54e None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:51:22.314513 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2d0b51-c83f-4531-994a-a0d42106a54e None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:51:22.314908 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2d0b51-c83f-4531-994a-a0d42106a54e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:51:22.322110 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ee4bfe-123c-4961-b29a-9a21b57c0af2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 10:51:22.327037 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application [None req-c6ee4bfe-123c-4961-b29a-9a21b57c0af2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 26 10:51:22.327037 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 10:51:22.327037 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 10:51:22.327037 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 10:51:22.327037 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 10:51:22.327037 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 10:51:22.327037 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 10:51:22.327037 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 10:51:22.327037 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 10:51:22.327037 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 10:51:22.327037 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 10:51:22.327037 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 10:51:22.327037 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 10:51:22.327037 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 10:51:22.327037 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 10:51:22.327037 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 10:51:22.327950 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 10:51:22.327950 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 10:51:22.327950 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 10:51:22.327950 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 10:51:22.327950 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 10:51:22.327950 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 10:51:22.327950 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 10:51:22.327950 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 10:51:22.327950 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 10:51:22.327950 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 10:51:22.327950 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 10:51:22.327950 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application with Lock( Aug 26 10:51:22.327950 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 10:51:22.327950 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._enter() Aug 26 10:51:22.327950 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 10:51:22.327950 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 10:51:22.328736 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 10:51:22.328736 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.creator() Aug 26 10:51:22.328736 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 10:51:22.328736 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application created_value = creator( Aug 26 10:51:22.328736 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 10:51:22.328736 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 10:51:22.328736 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 10:51:22.328736 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 10:51:22.328736 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 10:51:22.328736 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 10:51:22.328736 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 26 10:51:22.328736 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Aug 26 10:51:22.328736 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 96/186] 10.222.0.40 () {56 vars in 1096 bytes} [Fri Aug 26 10:51:22 2022] GET /identity/v3/roles/admin => generated 83 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 10:51:22.329397 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ee4bfe-123c-4961-b29a-9a21b57c0af2 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:51:22.335773 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-8e2d0b51-c83f-4531-994a-a0d42106a54e None None] MFA Rules not processed for user `ca155a48e2024c32af3fd2e19e5fac11`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 10:51:22.336977 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-8e2d0b51-c83f-4531-994a-a0d42106a54e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 10:51:22.412077 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 91/187] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 26 10:51:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 10:51:22.412965 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2d0b51-c83f-4531-994a-a0d42106a54e None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:51:22.440594 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ee4bfe-123c-4961-b29a-9a21b57c0af2 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:51:22.452216 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:51:22.478769 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f15b930-f1e2-4d31-bd6e-6d40987c2e2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:51:22.479695 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f15b930-f1e2-4d31-bd6e-6d40987c2e2b None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:51:22.480144 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f15b930-f1e2-4d31-bd6e-6d40987c2e2b None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:51:22.480611 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f15b930-f1e2-4d31-bd6e-6d40987c2e2b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:51:22.481696 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f15b930-f1e2-4d31-bd6e-6d40987c2e2b None admin] RBAC: Adding query filter params (role.id=1f97e4602e2e4d6498913a81f9ae8d6a, scope.project.id=da070dea52de4d71952ed50827d44a1c, user.id=ca155a48e2024c32af3fd2e19e5fac11) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 10:51:22.482095 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f15b930-f1e2-4d31-bd6e-6d40987c2e2b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:51:22.483787 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f15b930-f1e2-4d31-bd6e-6d40987c2e2b None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:51:22.498014 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 97/188] 10.222.0.40 () {56 vars in 1374 bytes} [Fri Aug 26 10:51:22 2022] GET /identity/v3/role_assignments?user.id=ca155a48e2024c32af3fd2e19e5fac11&scope.project.id=da070dea52de4d71952ed50827d44a1c&role.id=1f97e4602e2e4d6498913a81f9ae8d6a => generated 608 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 10:51:22.498014 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f15b930-f1e2-4d31-bd6e-6d40987c2e2b None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:51:22.513137 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:51:22.548241 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2d0b51-c83f-4531-994a-a0d42106a54e None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:51:22.591053 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10193626-7808-4e1e-863e-62486036e98e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:51:22.591821 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10193626-7808-4e1e-863e-62486036e98e None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:51:22.592217 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10193626-7808-4e1e-863e-62486036e98e None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:51:22.592719 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10193626-7808-4e1e-863e-62486036e98e None admin] PATH_INFO: `/v3/domains/default` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:51:22.594051 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10193626-7808-4e1e-863e-62486036e98e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:51:22.595748 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10193626-7808-4e1e-863e-62486036e98e None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:51:22.598096 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 92/189] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Aug 26 10:51:22 2022] GET /identity/v3/domains/default => generated 199 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 10:51:22.602925 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10193626-7808-4e1e-863e-62486036e98e None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:51:22.631783 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f15b930-f1e2-4d31-bd6e-6d40987c2e2b None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:51:22.676824 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95610aa5-c680-4a56-bcda-6f3eb60d841e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:51:22.677417 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95610aa5-c680-4a56-bcda-6f3eb60d841e None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:51:22.677698 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95610aa5-c680-4a56-bcda-6f3eb60d841e None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:51:22.677991 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95610aa5-c680-4a56-bcda-6f3eb60d841e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:51:22.678727 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95610aa5-c680-4a56-bcda-6f3eb60d841e None admin] RBAC: Adding query filter params (role.id=9c439f898126444f92abb01910f1d02b, scope.project.id=da070dea52de4d71952ed50827d44a1c, user.id=514ea0d902e04c41a548d79008a6d68d) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 10:51:22.679089 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95610aa5-c680-4a56-bcda-6f3eb60d841e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:51:22.680541 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95610aa5-c680-4a56-bcda-6f3eb60d841e None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:51:22.689794 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 98/190] 10.222.0.40 () {56 vars in 1374 bytes} [Fri Aug 26 10:51:22 2022] GET /identity/v3/role_assignments?user.id=514ea0d902e04c41a548d79008a6d68d&scope.project.id=da070dea52de4d71952ed50827d44a1c&role.id=9c439f898126444f92abb01910f1d02b => generated 608 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 10:51:22.689794 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95610aa5-c680-4a56-bcda-6f3eb60d841e None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:51:22.717758 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:51:22.728808 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10193626-7808-4e1e-863e-62486036e98e None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:51:22.786029 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1405cd04-e402-4b24-9b6d-1e36061b4614 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:51:22.787365 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1405cd04-e402-4b24-9b6d-1e36061b4614 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:51:22.787853 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1405cd04-e402-4b24-9b6d-1e36061b4614 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:51:22.788126 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1405cd04-e402-4b24-9b6d-1e36061b4614 None admin] PATH_INFO: `/v3/roles` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:51:22.789026 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1405cd04-e402-4b24-9b6d-1e36061b4614 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 10:51:22.789333 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1405cd04-e402-4b24-9b6d-1e36061b4614 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:51:22.791838 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1405cd04-e402-4b24-9b6d-1e36061b4614 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:51:22.801383 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:51:22.802695 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 93/191] 10.222.0.40 () {56 vars in 1099 bytes} [Fri Aug 26 10:51:22 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 10:51:22.803878 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1405cd04-e402-4b24-9b6d-1e36061b4614 None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:51:22.829658 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95610aa5-c680-4a56-bcda-6f3eb60d841e None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:51:22.875910 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9cddae-0b81-49e4-9def-69b6017263c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:51:22.876553 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9cddae-0b81-49e4-9def-69b6017263c3 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:51:22.876867 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9cddae-0b81-49e4-9def-69b6017263c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:51:22.877161 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9cddae-0b81-49e4-9def-69b6017263c3 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:51:22.890333 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff9cddae-0b81-49e4-9def-69b6017263c3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 10:51:22.896220 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application [None req-ff9cddae-0b81-49e4-9def-69b6017263c3 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 26 10:51:22.896220 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 10:51:22.896220 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 10:51:22.896220 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 10:51:22.896220 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 10:51:22.896220 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 10:51:22.896220 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 10:51:22.896220 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 10:51:22.896220 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 10:51:22.896220 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 10:51:22.896220 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 10:51:22.896220 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 10:51:22.896220 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 10:51:22.896220 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 10:51:22.896220 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 10:51:22.896220 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 10:51:22.897092 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 10:51:22.897092 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 10:51:22.897092 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 10:51:22.897092 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 10:51:22.897092 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 10:51:22.897092 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 10:51:22.897092 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 10:51:22.897092 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 10:51:22.897092 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 10:51:22.897092 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 10:51:22.897092 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 10:51:22.897092 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application with Lock( Aug 26 10:51:22.897092 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 10:51:22.897092 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._enter() Aug 26 10:51:22.897092 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 10:51:22.897092 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 10:51:22.897858 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 10:51:22.897858 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.creator() Aug 26 10:51:22.897858 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 10:51:22.897858 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application created_value = creator( Aug 26 10:51:22.897858 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 10:51:22.897858 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 10:51:22.897858 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 10:51:22.897858 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 10:51:22.897858 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 10:51:22.897858 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 10:51:22.897858 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 26 10:51:22.897858 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Aug 26 10:51:22.897858 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 99/192] 10.222.0.40 () {56 vars in 1114 bytes} [Fri Aug 26 10:51:22 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 10:51:22.898557 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9cddae-0b81-49e4-9def-69b6017263c3 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:51:22.939679 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1405cd04-e402-4b24-9b6d-1e36061b4614 None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:51:22.974338 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bcbf541-f6ed-4127-8809-9e926b202b69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:51:22.974944 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcbf541-f6ed-4127-8809-9e926b202b69 None admin] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:51:22.975222 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcbf541-f6ed-4127-8809-9e926b202b69 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:51:22.975492 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcbf541-f6ed-4127-8809-9e926b202b69 None admin] PATH_INFO: `/v3/users` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:51:22.976126 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bcbf541-f6ed-4127-8809-9e926b202b69 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:51:22.977561 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bcbf541-f6ed-4127-8809-9e926b202b69 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:51:22.998498 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 94/193] 10.222.0.40 () {60 vars in 1132 bytes} [Fri Aug 26 10:51:22 2022] POST /identity/v3/users => generated 297 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 10:51:22.999317 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bcbf541-f6ed-4127-8809-9e926b202b69 None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:51:23.016509 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:51:23.020467 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9cddae-0b81-49e4-9def-69b6017263c3 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:51:23.058813 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f3e4c4-a900-443f-a72a-156db3136eb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:51:23.059618 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f3e4c4-a900-443f-a72a-156db3136eb5 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:51:23.059899 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f3e4c4-a900-443f-a72a-156db3136eb5 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:51:23.060171 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f3e4c4-a900-443f-a72a-156db3136eb5 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:51:23.063366 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:51:23.073022 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f3e4c4-a900-443f-a72a-156db3136eb5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 10:51:23.078467 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application [None req-b9f3e4c4-a900-443f-a72a-156db3136eb5 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 26 10:51:23.078467 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 10:51:23.078467 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 10:51:23.078467 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 10:51:23.078467 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 10:51:23.078467 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 10:51:23.078467 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 10:51:23.078467 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 10:51:23.078467 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 10:51:23.078467 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 10:51:23.078467 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 10:51:23.078467 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 10:51:23.078467 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 10:51:23.078467 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 10:51:23.078467 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 10:51:23.078467 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 10:51:23.079638 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 10:51:23.079638 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 10:51:23.079638 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 10:51:23.079638 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 10:51:23.079638 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 10:51:23.079638 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 10:51:23.079638 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 10:51:23.079638 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 10:51:23.079638 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 10:51:23.079638 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 10:51:23.079638 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 10:51:23.079638 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application with Lock( Aug 26 10:51:23.079638 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 10:51:23.079638 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._enter() Aug 26 10:51:23.079638 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 10:51:23.079638 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 10:51:23.081022 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 10:51:23.081022 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.creator() Aug 26 10:51:23.081022 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 10:51:23.081022 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application created_value = creator( Aug 26 10:51:23.081022 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 10:51:23.081022 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 10:51:23.081022 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 10:51:23.081022 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 10:51:23.081022 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 10:51:23.081022 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 10:51:23.081022 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 26 10:51:23.081022 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Aug 26 10:51:23.081022 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 100/194] 10.222.0.40 () {56 vars in 1099 bytes} [Fri Aug 26 10:51:22 2022] GET /identity/v3/roles/reader => generated 84 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 10:51:23.081694 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f3e4c4-a900-443f-a72a-156db3136eb5 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:51:23.126102 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bcbf541-f6ed-4127-8809-9e926b202b69 None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:51:23.169059 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b1e24f-9b58-4da8-812e-3b6831cd4edb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:51:23.169709 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b1e24f-9b58-4da8-812e-3b6831cd4edb None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:51:23.170005 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b1e24f-9b58-4da8-812e-3b6831cd4edb None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:51:23.170310 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b1e24f-9b58-4da8-812e-3b6831cd4edb None admin] PATH_INFO: `/v3/users/00deb7d8c98747c6a68e18b5485c8a02` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:51:23.185695 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b1e24f-9b58-4da8-812e-3b6831cd4edb None admin] RBAC: Authorizing `identity:get_user(user_id=00deb7d8c98747c6a68e18b5485c8a02)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:51:23.187196 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b1e24f-9b58-4da8-812e-3b6831cd4edb None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:51:23.188868 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 95/195] 10.222.0.40 () {56 vars in 1177 bytes} [Fri Aug 26 10:51:22 2022] GET /identity/v3/users/00deb7d8c98747c6a68e18b5485c8a02 => generated 283 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 10:51:23.190749 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b1e24f-9b58-4da8-812e-3b6831cd4edb None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:51:23.203243 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f3e4c4-a900-443f-a72a-156db3136eb5 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:51:23.251595 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f057a0e6-a782-46fe-b98a-ad039f39001c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:51:23.252217 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f057a0e6-a782-46fe-b98a-ad039f39001c None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:51:23.252543 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f057a0e6-a782-46fe-b98a-ad039f39001c None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:51:23.252906 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f057a0e6-a782-46fe-b98a-ad039f39001c None admin] PATH_INFO: `/v3/roles` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:51:23.253581 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f057a0e6-a782-46fe-b98a-ad039f39001c None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 10:51:23.253881 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f057a0e6-a782-46fe-b98a-ad039f39001c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:51:23.255317 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f057a0e6-a782-46fe-b98a-ad039f39001c None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:51:23.265298 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 101/196] 10.222.0.40 () {56 vars in 1111 bytes} [Fri Aug 26 10:51:23 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 10:51:23.265298 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f057a0e6-a782-46fe-b98a-ad039f39001c None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:51:23.278202 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:51:23.314507 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b1e24f-9b58-4da8-812e-3b6831cd4edb None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:51:23.332650 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:51:23.360622 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c6c0a2-ff5f-4e46-b671-bc8c92879978 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:51:23.361242 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c6c0a2-ff5f-4e46-b671-bc8c92879978 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:51:23.361512 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c6c0a2-ff5f-4e46-b671-bc8c92879978 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:51:23.361822 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c6c0a2-ff5f-4e46-b671-bc8c92879978 None admin] PATH_INFO: `/v3/roles` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:51:23.362701 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c6c0a2-ff5f-4e46-b671-bc8c92879978 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 10:51:23.363004 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c6c0a2-ff5f-4e46-b671-bc8c92879978 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:51:23.364388 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c6c0a2-ff5f-4e46-b671-bc8c92879978 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:51:23.377849 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 96/197] 10.222.0.40 () {56 vars in 1101 bytes} [Fri Aug 26 10:51:23 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 10:51:23.379129 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c6c0a2-ff5f-4e46-b671-bc8c92879978 None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:51:23.391671 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f057a0e6-a782-46fe-b98a-ad039f39001c None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:51:23.428507 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dafc539-8c3e-409a-8756-be7d489751c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:51:23.429123 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dafc539-8c3e-409a-8756-be7d489751c3 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:51:23.429394 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dafc539-8c3e-409a-8756-be7d489751c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:51:23.429663 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dafc539-8c3e-409a-8756-be7d489751c3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:51:23.500038 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dafc539-8c3e-409a-8756-be7d489751c3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:51:23.501255 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dafc539-8c3e-409a-8756-be7d489751c3 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:51:23.510960 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c6c0a2-ff5f-4e46-b671-bc8c92879978 None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:51:23.559004 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4821351-628b-4dc9-9833-77dc0f962e0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:51:23.559899 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4821351-628b-4dc9-9833-77dc0f962e0d None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:51:23.560315 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4821351-628b-4dc9-9833-77dc0f962e0d None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:51:23.560937 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:51:23.561236 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4821351-628b-4dc9-9833-77dc0f962e0d None admin] PATH_INFO: `/v3/users/00deb7d8c98747c6a68e18b5485c8a02` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:51:23.563290 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4821351-628b-4dc9-9833-77dc0f962e0d None admin] RBAC: Authorizing `identity:get_user(user_id=00deb7d8c98747c6a68e18b5485c8a02)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:51:23.565571 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4821351-628b-4dc9-9833-77dc0f962e0d None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:51:23.567579 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 97/198] 10.222.0.40 () {56 vars in 1177 bytes} [Fri Aug 26 10:51:23 2022] GET /identity/v3/users/00deb7d8c98747c6a68e18b5485c8a02 => generated 283 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 10:51:23.569232 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4821351-628b-4dc9-9833-77dc0f962e0d None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:51:23.572066 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:51:23.611475 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 102/199] 10.222.0.40 () {58 vars in 1282 bytes} [Fri Aug 26 10:51:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 26 10:51:23.612362 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dafc539-8c3e-409a-8756-be7d489751c3 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:51:23.684701 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4821351-628b-4dc9-9833-77dc0f962e0d None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:51:23.722712 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dafc539-8c3e-409a-8756-be7d489751c3 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:51:23.729737 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21fd298-8df6-4bbf-b845-54e3c0c48c55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:51:23.731097 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21fd298-8df6-4bbf-b845-54e3c0c48c55 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:51:23.731589 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21fd298-8df6-4bbf-b845-54e3c0c48c55 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:51:23.732010 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21fd298-8df6-4bbf-b845-54e3c0c48c55 None admin] PATH_INFO: `/v3/users/a19a4f7cc5bd49409158ab71314d727d` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:51:23.734425 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a21fd298-8df6-4bbf-b845-54e3c0c48c55 None admin] RBAC: Authorizing `identity:get_user(user_id=a19a4f7cc5bd49409158ab71314d727d)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:51:23.736256 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a21fd298-8df6-4bbf-b845-54e3c0c48c55 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:51:23.739698 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 98/200] 10.222.0.40 () {56 vars in 1177 bytes} [Fri Aug 26 10:51:23 2022] GET /identity/v3/users/a19a4f7cc5bd49409158ab71314d727d => generated 289 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 10:51:23.740732 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21fd298-8df6-4bbf-b845-54e3c0c48c55 None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:51:23.765357 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f41390-90a8-4412-866d-b4cb697ec21a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:51:23.765891 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f41390-90a8-4412-866d-b4cb697ec21a None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:51:23.766211 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f41390-90a8-4412-866d-b4cb697ec21a None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:51:23.766484 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f41390-90a8-4412-866d-b4cb697ec21a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:51:23.797865 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:51:23.834484 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:51:23.842305 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13f41390-90a8-4412-866d-b4cb697ec21a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:51:23.843794 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13f41390-90a8-4412-866d-b4cb697ec21a None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:51:23.883763 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21fd298-8df6-4bbf-b845-54e3c0c48c55 None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:51:23.930264 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3a6ab8-4ef8-43d1-9560-209245c90ae3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:51:23.930904 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3a6ab8-4ef8-43d1-9560-209245c90ae3 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:51:23.931544 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3a6ab8-4ef8-43d1-9560-209245c90ae3 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:51:23.931828 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3a6ab8-4ef8-43d1-9560-209245c90ae3 None admin] PATH_INFO: `/v3/projects/15667f91e7f6473cb9aab41830d80f5e` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:51:23.933297 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd3a6ab8-4ef8-43d1-9560-209245c90ae3 None admin] RBAC: Authorizing `identity:get_project(project_id=15667f91e7f6473cb9aab41830d80f5e)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:51:23.935199 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd3a6ab8-4ef8-43d1-9560-209245c90ae3 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:51:23.937632 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 99/201] 10.222.0.40 () {56 vars in 1186 bytes} [Fri Aug 26 10:51:23 2022] GET /identity/v3/projects/15667f91e7f6473cb9aab41830d80f5e => generated 302 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 10:51:23.938533 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3a6ab8-4ef8-43d1-9560-209245c90ae3 None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:51:23.959025 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 103/202] 10.222.0.40 () {58 vars in 1282 bytes} [Fri Aug 26 10:51:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 26 10:51:23.959943 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f41390-90a8-4412-866d-b4cb697ec21a None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:51:24.018899 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:51:24.061987 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f41390-90a8-4412-866d-b4cb697ec21a None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:51:24.062175 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3a6ab8-4ef8-43d1-9560-209245c90ae3 None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:51:24.097614 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499550aa-a0b0-472a-a679-3836f2dcf3f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:51:24.098391 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499550aa-a0b0-472a-a679-3836f2dcf3f7 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:51:24.098741 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499550aa-a0b0-472a-a679-3836f2dcf3f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:51:24.099086 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499550aa-a0b0-472a-a679-3836f2dcf3f7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:51:24.099885 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-499550aa-a0b0-472a-a679-3836f2dcf3f7 None admin] RBAC: Adding query filter params (role.id=1f97e4602e2e4d6498913a81f9ae8d6a, scope.project.id=15667f91e7f6473cb9aab41830d80f5e, user.id=00deb7d8c98747c6a68e18b5485c8a02) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 10:51:24.100266 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-499550aa-a0b0-472a-a679-3836f2dcf3f7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:51:24.101778 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-499550aa-a0b0-472a-a679-3836f2dcf3f7 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:51:24.103928 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e021e91-febd-4d95-a236-bdea356e936a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:51:24.104728 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e021e91-febd-4d95-a236-bdea356e936a None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:51:24.105073 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e021e91-febd-4d95-a236-bdea356e936a None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:51:24.105410 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e021e91-febd-4d95-a236-bdea356e936a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:51:24.107497 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 104/203] 10.222.0.40 () {56 vars in 1374 bytes} [Fri Aug 26 10:51:23 2022] GET /identity/v3/role_assignments?user.id=00deb7d8c98747c6a68e18b5485c8a02&scope.project.id=15667f91e7f6473cb9aab41830d80f5e&role.id=1f97e4602e2e4d6498913a81f9ae8d6a => generated 259 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 10:51:24.108629 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499550aa-a0b0-472a-a679-3836f2dcf3f7 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:51:24.122368 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:51:24.224526 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499550aa-a0b0-472a-a679-3836f2dcf3f7 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:51:24.249328 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e021e91-febd-4d95-a236-bdea356e936a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:51:24.255025 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e021e91-febd-4d95-a236-bdea356e936a None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:51:24.258377 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:51:24.266676 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9efc2d4c-cf6f-488a-a998-d36970a8053b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:51:24.267464 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efc2d4c-cf6f-488a-a998-d36970a8053b None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:51:24.268076 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efc2d4c-cf6f-488a-a998-d36970a8053b None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:51:24.268726 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efc2d4c-cf6f-488a-a998-d36970a8053b None admin] PATH_INFO: `/v3/projects/15667f91e7f6473cb9aab41830d80f5e` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:51:24.270269 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9efc2d4c-cf6f-488a-a998-d36970a8053b None admin] RBAC: Authorizing `identity:get_project(project_id=15667f91e7f6473cb9aab41830d80f5e)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:51:24.271786 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9efc2d4c-cf6f-488a-a998-d36970a8053b None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:51:24.274166 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 105/204] 10.222.0.40 () {56 vars in 1186 bytes} [Fri Aug 26 10:51:24 2022] GET /identity/v3/projects/15667f91e7f6473cb9aab41830d80f5e => generated 302 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 10:51:24.281086 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9efc2d4c-cf6f-488a-a998-d36970a8053b None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:51:24.438905 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 100/205] 10.222.0.40 () {58 vars in 1282 bytes} [Fri Aug 26 10:51:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 501 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 26 10:51:24.447704 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e021e91-febd-4d95-a236-bdea356e936a None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:51:24.496307 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9efc2d4c-cf6f-488a-a998-d36970a8053b None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:51:24.504793 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:51:24.556830 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdec3a56-649f-4e35-b4ed-e3b02a2359d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:51:24.557469 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdec3a56-649f-4e35-b4ed-e3b02a2359d5 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:51:24.557757 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdec3a56-649f-4e35-b4ed-e3b02a2359d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:51:24.558048 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdec3a56-649f-4e35-b4ed-e3b02a2359d5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:51:24.558783 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdec3a56-649f-4e35-b4ed-e3b02a2359d5 None admin] RBAC: Adding query filter params (role.id=9c439f898126444f92abb01910f1d02b, scope.project.id=15667f91e7f6473cb9aab41830d80f5e, user.id=00deb7d8c98747c6a68e18b5485c8a02) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 10:51:24.559086 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdec3a56-649f-4e35-b4ed-e3b02a2359d5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:51:24.560801 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdec3a56-649f-4e35-b4ed-e3b02a2359d5 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:51:24.572439 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 106/206] 10.222.0.40 () {56 vars in 1374 bytes} [Fri Aug 26 10:51:24 2022] GET /identity/v3/role_assignments?user.id=00deb7d8c98747c6a68e18b5485c8a02&scope.project.id=15667f91e7f6473cb9aab41830d80f5e&role.id=9c439f898126444f92abb01910f1d02b => generated 259 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 10:51:24.580353 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e021e91-febd-4d95-a236-bdea356e936a None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:51:24.618799 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6003f32-b470-46f1-878e-e278c6e5bd4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:51:24.619459 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6003f32-b470-46f1-878e-e278c6e5bd4b None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:51:24.619737 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6003f32-b470-46f1-878e-e278c6e5bd4b None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:51:24.620010 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6003f32-b470-46f1-878e-e278c6e5bd4b None admin] PATH_INFO: `/v3/projects/da070dea52de4d71952ed50827d44a1c` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:51:24.621240 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6003f32-b470-46f1-878e-e278c6e5bd4b None admin] RBAC: Authorizing `identity:get_project(project_id=da070dea52de4d71952ed50827d44a1c)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:51:24.622763 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6003f32-b470-46f1-878e-e278c6e5bd4b None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:51:24.624841 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 101/207] 10.222.0.40 () {56 vars in 1186 bytes} [Fri Aug 26 10:51:24 2022] GET /identity/v3/projects/da070dea52de4d71952ed50827d44a1c => generated 298 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 10:51:24.631270 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdec3a56-649f-4e35-b4ed-e3b02a2359d5 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:51:24.733232 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:51:24.787292 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdec3a56-649f-4e35-b4ed-e3b02a2359d5 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:51:24.825141 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae575052-f7f9-4449-893a-cc97893d10e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:51:24.825677 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae575052-f7f9-4449-893a-cc97893d10e7 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:51:24.825953 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae575052-f7f9-4449-893a-cc97893d10e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:51:24.826251 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae575052-f7f9-4449-893a-cc97893d10e7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:51:24.826975 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae575052-f7f9-4449-893a-cc97893d10e7 None admin] RBAC: Adding query filter params (role.id=dae11e55a3f343ca9954bc6a1e748eed, scope.project.id=da070dea52de4d71952ed50827d44a1c, user.id=a19a4f7cc5bd49409158ab71314d727d) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 10:51:24.827278 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae575052-f7f9-4449-893a-cc97893d10e7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:51:24.828726 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae575052-f7f9-4449-893a-cc97893d10e7 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:51:24.838885 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 107/208] 10.222.0.40 () {56 vars in 1374 bytes} [Fri Aug 26 10:51:24 2022] GET /identity/v3/role_assignments?user.id=a19a4f7cc5bd49409158ab71314d727d&scope.project.id=da070dea52de4d71952ed50827d44a1c&role.id=dae11e55a3f343ca9954bc6a1e748eed => generated 608 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 10:51:25.187055 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78de481c-4b97-4bb2-b8cc-80b84e0e1ca8 None None] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:51:25.187457 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78de481c-4b97-4bb2-b8cc-80b84e0e1ca8 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:51:25.187630 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78de481c-4b97-4bb2-b8cc-80b84e0e1ca8 None None] PATH_INFO: `/` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:51:25.189230 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 102/209] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 26 10:51:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 10:51:25.194698 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3316128d-5f8f-4f2f-aa3e-ff01acfbb9e1 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:51:25.194983 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3316128d-5f8f-4f2f-aa3e-ff01acfbb9e1 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:51:25.195256 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3316128d-5f8f-4f2f-aa3e-ff01acfbb9e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:51:25.217256 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-3316128d-5f8f-4f2f-aa3e-ff01acfbb9e1 None None] MFA Rules not processed for user `ca155a48e2024c32af3fd2e19e5fac11`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 10:51:25.218579 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-3316128d-5f8f-4f2f-aa3e-ff01acfbb9e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 10:51:25.283142 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc72811-f5a1-4bc8-8aab-55eb8894e9ce None None] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:51:25.283473 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc72811-f5a1-4bc8-8aab-55eb8894e9ce None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:51:25.284388 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc72811-f5a1-4bc8-8aab-55eb8894e9ce None None] PATH_INFO: `/` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:51:25.285849 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 103/210] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 26 10:51:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 10:51:25.293572 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f0cc32-7b8b-48ca-b560-f47a9e0e56c2 None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:51:25.293852 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f0cc32-7b8b-48ca-b560-f47a9e0e56c2 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:51:25.294150 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f0cc32-7b8b-48ca-b560-f47a9e0e56c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:51:25.298163 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 108/211] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 26 10:51:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 10:51:25.319003 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-87f0cc32-7b8b-48ca-b560-f47a9e0e56c2 None None] MFA Rules not processed for user `ca155a48e2024c32af3fd2e19e5fac11`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 10:51:25.320213 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-87f0cc32-7b8b-48ca-b560-f47a9e0e56c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 10:51:25.405609 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 104/212] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 26 10:51:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 10:51:25.586364 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3316128d-5f8f-4f2f-aa3e-ff01acfbb9e1 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:51:25.640872 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:51:25.692758 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3316128d-5f8f-4f2f-aa3e-ff01acfbb9e1 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:51:25.706634 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f0cc32-7b8b-48ca-b560-f47a9e0e56c2 None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:51:25.728447 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e1dc43-08df-4fee-8518-2cb3da5949d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:51:25.729534 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e1dc43-08df-4fee-8518-2cb3da5949d2 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:51:25.729809 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e1dc43-08df-4fee-8518-2cb3da5949d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:51:25.730365 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e1dc43-08df-4fee-8518-2cb3da5949d2 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:51:25.742687 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:51:25.744134 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e1dc43-08df-4fee-8518-2cb3da5949d2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 10:51:25.749308 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application [None req-b4e1dc43-08df-4fee-8518-2cb3da5949d2 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 26 10:51:25.749308 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 10:51:25.749308 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 10:51:25.749308 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 10:51:25.749308 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 10:51:25.749308 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 10:51:25.749308 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 10:51:25.749308 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 10:51:25.749308 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 10:51:25.749308 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 10:51:25.749308 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 10:51:25.749308 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 10:51:25.749308 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 10:51:25.749308 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 10:51:25.749308 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 10:51:25.749308 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 10:51:25.750270 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 10:51:25.750270 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 10:51:25.750270 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 10:51:25.750270 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 10:51:25.750270 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 10:51:25.750270 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 10:51:25.750270 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 10:51:25.750270 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 10:51:25.750270 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 10:51:25.750270 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 10:51:25.750270 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 10:51:25.750270 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application with Lock( Aug 26 10:51:25.750270 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 10:51:25.750270 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._enter() Aug 26 10:51:25.750270 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 10:51:25.750270 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 10:51:25.751052 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 10:51:25.751052 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.creator() Aug 26 10:51:25.751052 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 10:51:25.751052 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application created_value = creator( Aug 26 10:51:25.751052 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 10:51:25.751052 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 10:51:25.751052 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 10:51:25.751052 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 10:51:25.751052 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 10:51:25.751052 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 10:51:25.751052 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 26 10:51:25.751052 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Aug 26 10:51:25.751649 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 109/213] 10.222.0.40 () {56 vars in 1099 bytes} [Fri Aug 26 10:51:25 2022] GET /identity/v3/roles/member => generated 84 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 10:51:25.757685 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e1dc43-08df-4fee-8518-2cb3da5949d2 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:51:25.826348 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f0cc32-7b8b-48ca-b560-f47a9e0e56c2 None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:51:25.868122 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e1dc43-08df-4fee-8518-2cb3da5949d2 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:51:25.883360 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e8b07d-c806-4956-8493-fc5a4c486617 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:51:25.884514 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e8b07d-c806-4956-8493-fc5a4c486617 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:51:25.884986 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e8b07d-c806-4956-8493-fc5a4c486617 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:51:25.885602 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e8b07d-c806-4956-8493-fc5a4c486617 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:51:25.887613 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e8b07d-c806-4956-8493-fc5a4c486617 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:51:25.889418 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e8b07d-c806-4956-8493-fc5a4c486617 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:51:25.890355 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:51:25.892160 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 105/214] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Aug 26 10:51:25 2022] GET /identity/v3/domains/default => generated 199 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 10:51:25.898118 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e8b07d-c806-4956-8493-fc5a4c486617 None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:51:25.912269 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7ab217-3c9b-44b8-bad1-925698731083 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:51:25.913201 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7ab217-3c9b-44b8-bad1-925698731083 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:51:25.913700 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7ab217-3c9b-44b8-bad1-925698731083 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:51:25.914196 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7ab217-3c9b-44b8-bad1-925698731083 None admin] PATH_INFO: `/v3/roles` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:51:25.915086 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c7ab217-3c9b-44b8-bad1-925698731083 None admin] RBAC: Adding query filter params (name=member) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 10:51:25.915561 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c7ab217-3c9b-44b8-bad1-925698731083 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:51:25.917202 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c7ab217-3c9b-44b8-bad1-925698731083 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:51:25.927030 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 110/215] 10.222.0.40 () {56 vars in 1101 bytes} [Fri Aug 26 10:51:25 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 10:51:25.932912 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7ab217-3c9b-44b8-bad1-925698731083 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:51:26.013680 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e8b07d-c806-4956-8493-fc5a4c486617 None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:51:26.016133 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:51:26.042621 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7ab217-3c9b-44b8-bad1-925698731083 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:51:26.052069 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c258471f-e23b-4b16-880c-2f7554acb6eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:51:26.052859 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c258471f-e23b-4b16-880c-2f7554acb6eb None admin] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:51:26.053338 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c258471f-e23b-4b16-880c-2f7554acb6eb None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:51:26.053794 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c258471f-e23b-4b16-880c-2f7554acb6eb None admin] PATH_INFO: `/v3/users` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:51:26.054764 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c258471f-e23b-4b16-880c-2f7554acb6eb None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:51:26.056377 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c258471f-e23b-4b16-880c-2f7554acb6eb None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:51:26.081144 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 106/216] 10.222.0.40 () {60 vars in 1132 bytes} [Fri Aug 26 10:51:25 2022] POST /identity/v3/users => generated 297 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 10:51:26.094968 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31381bc4-4451-48bc-990d-f62d77949d00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:51:26.095686 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31381bc4-4451-48bc-990d-f62d77949d00 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:51:26.095968 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31381bc4-4451-48bc-990d-f62d77949d00 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:51:26.096336 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31381bc4-4451-48bc-990d-f62d77949d00 None admin] PATH_INFO: `/v3/users/c59df4b4751540dc8126c66c4843c7a2` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:51:26.114562 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31381bc4-4451-48bc-990d-f62d77949d00 None admin] RBAC: Authorizing `identity:get_user(user_id=c59df4b4751540dc8126c66c4843c7a2)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:51:26.116067 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31381bc4-4451-48bc-990d-f62d77949d00 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:51:26.118502 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 111/217] 10.222.0.40 () {56 vars in 1177 bytes} [Fri Aug 26 10:51:25 2022] GET /identity/v3/users/c59df4b4751540dc8126c66c4843c7a2 => generated 297 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 10:51:26.123931 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c258471f-e23b-4b16-880c-2f7554acb6eb None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:51:26.243632 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c258471f-e23b-4b16-880c-2f7554acb6eb None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:51:26.279961 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c934c6e1-ae0c-4fdc-b7b8-c1f788e7ef9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:51:26.280588 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c934c6e1-ae0c-4fdc-b7b8-c1f788e7ef9a None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:51:26.280865 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c934c6e1-ae0c-4fdc-b7b8-c1f788e7ef9a None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:51:26.281136 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c934c6e1-ae0c-4fdc-b7b8-c1f788e7ef9a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:51:26.311636 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:51:26.357168 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c934c6e1-ae0c-4fdc-b7b8-c1f788e7ef9a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:51:26.358967 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c934c6e1-ae0c-4fdc-b7b8-c1f788e7ef9a None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:51:26.400627 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf80d4f-05c6-430c-bcb8-69bf402a8efa None None] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:51:26.400979 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf80d4f-05c6-430c-bcb8-69bf402a8efa None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:51:26.401201 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf80d4f-05c6-430c-bcb8-69bf402a8efa None None] PATH_INFO: `/` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:51:26.402600 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 112/218] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 26 10:51:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 10:51:26.407978 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb81ebc9-fca8-48d0-8239-9ac680937271 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:51:26.408565 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb81ebc9-fca8-48d0-8239-9ac680937271 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:51:26.409032 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb81ebc9-fca8-48d0-8239-9ac680937271 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:51:26.428438 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-fb81ebc9-fca8-48d0-8239-9ac680937271 None None] MFA Rules not processed for user `ca155a48e2024c32af3fd2e19e5fac11`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 10:51:26.429961 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-fb81ebc9-fca8-48d0-8239-9ac680937271 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 10:51:26.464363 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:51:26.481564 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 107/219] 10.222.0.40 () {58 vars in 1282 bytes} [Fri Aug 26 10:51:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 356 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 26 10:51:26.487916 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c934c6e1-ae0c-4fdc-b7b8-c1f788e7ef9a None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:51:26.528657 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 113/220] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 26 10:51:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 26 10:51:26.559735 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:51:26.605610 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c934c6e1-ae0c-4fdc-b7b8-c1f788e7ef9a None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:51:26.641723 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97f6d77-ea25-4bf6-af68-071eb692abe5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:51:26.642741 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97f6d77-ea25-4bf6-af68-071eb692abe5 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:51:26.644333 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97f6d77-ea25-4bf6-af68-071eb692abe5 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:51:26.644853 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97f6d77-ea25-4bf6-af68-071eb692abe5 None admin] PATH_INFO: `/v3/projects/15667f91e7f6473cb9aab41830d80f5e` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:51:26.646222 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d97f6d77-ea25-4bf6-af68-071eb692abe5 None admin] RBAC: Authorizing `identity:get_project(project_id=15667f91e7f6473cb9aab41830d80f5e)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:51:26.647874 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d97f6d77-ea25-4bf6-af68-071eb692abe5 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:51:26.652945 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 108/221] 10.222.0.40 () {56 vars in 1186 bytes} [Fri Aug 26 10:51:26 2022] GET /identity/v3/projects/15667f91e7f6473cb9aab41830d80f5e => generated 302 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 10:51:26.656532 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb81ebc9-fca8-48d0-8239-9ac680937271 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:51:26.780274 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb81ebc9-fca8-48d0-8239-9ac680937271 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:51:26.826617 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec03fa1-24cb-4882-969c-47d5c92c0c1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:51:26.827634 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec03fa1-24cb-4882-969c-47d5c92c0c1a None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:51:26.828186 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec03fa1-24cb-4882-969c-47d5c92c0c1a None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:51:26.828686 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec03fa1-24cb-4882-969c-47d5c92c0c1a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:51:26.829655 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec03fa1-24cb-4882-969c-47d5c92c0c1a None admin] RBAC: Adding query filter params (role.id=23e174ce743b46eabe618c212691da55, scope.project.id=15667f91e7f6473cb9aab41830d80f5e, user.id=c59df4b4751540dc8126c66c4843c7a2) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 10:51:26.830228 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec03fa1-24cb-4882-969c-47d5c92c0c1a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:51:26.831902 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec03fa1-24cb-4882-969c-47d5c92c0c1a None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:51:26.842194 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 114/222] 10.222.0.40 () {56 vars in 1374 bytes} [Fri Aug 26 10:51:26 2022] GET /identity/v3/role_assignments?user.id=c59df4b4751540dc8126c66c4843c7a2&scope.project.id=15667f91e7f6473cb9aab41830d80f5e&role.id=23e174ce743b46eabe618c212691da55 => generated 259 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 10:51:26.860693 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa8396d-ce09-48cc-802e-55af2d192a11 None None] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:51:26.861188 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa8396d-ce09-48cc-802e-55af2d192a11 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:51:26.861260 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa8396d-ce09-48cc-802e-55af2d192a11 None None] PATH_INFO: `/` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:51:26.863501 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 109/223] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 26 10:51:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 10:51:26.869184 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a2b8dc-c015-470b-92f4-572ee66590be None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:51:26.869492 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a2b8dc-c015-470b-92f4-572ee66590be None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:51:26.869735 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a2b8dc-c015-470b-92f4-572ee66590be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:51:26.893676 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa8396d-ce09-48cc-802e-55af2d192a11 None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:51:26.903128 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-60a2b8dc-c015-470b-92f4-572ee66590be None None] MFA Rules not processed for user `ca155a48e2024c32af3fd2e19e5fac11`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 10:51:26.904702 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-60a2b8dc-c015-470b-92f4-572ee66590be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 10:51:26.939005 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:51:27.029395 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 115/224] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 26 10:51:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 10:51:27.075228 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa8396d-ce09-48cc-802e-55af2d192a11 None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:51:27.123100 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:51:27.135569 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5757c24e-2787-464e-84cc-dd565859aede None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:51:27.136205 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5757c24e-2787-464e-84cc-dd565859aede None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:51:27.136479 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5757c24e-2787-464e-84cc-dd565859aede None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:51:27.136779 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5757c24e-2787-464e-84cc-dd565859aede None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:51:27.151596 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5757c24e-2787-464e-84cc-dd565859aede None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 10:51:27.156670 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application [None req-5757c24e-2787-464e-84cc-dd565859aede None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 26 10:51:27.156670 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 10:51:27.156670 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 10:51:27.156670 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 10:51:27.156670 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 10:51:27.156670 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 10:51:27.156670 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 10:51:27.156670 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 10:51:27.156670 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 10:51:27.156670 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 10:51:27.156670 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 10:51:27.156670 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 10:51:27.156670 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 10:51:27.156670 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 10:51:27.156670 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 10:51:27.156670 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 10:51:27.157530 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 10:51:27.157530 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 10:51:27.157530 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 10:51:27.157530 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 10:51:27.157530 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 10:51:27.157530 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 10:51:27.157530 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 10:51:27.157530 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 10:51:27.157530 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 10:51:27.157530 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 10:51:27.157530 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 10:51:27.157530 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application with Lock( Aug 26 10:51:27.157530 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 10:51:27.157530 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._enter() Aug 26 10:51:27.157530 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 10:51:27.157530 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 10:51:27.158352 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 10:51:27.158352 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.creator() Aug 26 10:51:27.158352 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 10:51:27.158352 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application created_value = creator( Aug 26 10:51:27.158352 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 10:51:27.158352 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 10:51:27.158352 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 10:51:27.158352 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 10:51:27.158352 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 10:51:27.158352 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 10:51:27.158352 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 26 10:51:27.158352 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Aug 26 10:51:27.158352 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 110/225] 10.222.0.40 () {56 vars in 1096 bytes} [Fri Aug 26 10:51:26 2022] GET /identity/v3/roles/admin => generated 83 bytes in 265 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 10:51:27.165672 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a2b8dc-c015-470b-92f4-572ee66590be None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:51:27.314319 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a2b8dc-c015-470b-92f4-572ee66590be None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:51:27.350125 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83dc82de-c95a-44ea-9c3e-6b1db818fbdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:51:27.350745 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83dc82de-c95a-44ea-9c3e-6b1db818fbdb None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:51:27.351024 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83dc82de-c95a-44ea-9c3e-6b1db818fbdb None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:51:27.351295 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83dc82de-c95a-44ea-9c3e-6b1db818fbdb None admin] PATH_INFO: `/v3/roles` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:51:27.351966 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83dc82de-c95a-44ea-9c3e-6b1db818fbdb None admin] RBAC: Adding query filter params (name=admin) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 10:51:27.352269 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83dc82de-c95a-44ea-9c3e-6b1db818fbdb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:51:27.353648 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83dc82de-c95a-44ea-9c3e-6b1db818fbdb None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:51:27.357948 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5757c24e-2787-464e-84cc-dd565859aede None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:51:27.363498 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 116/226] 10.222.0.40 () {56 vars in 1099 bytes} [Fri Aug 26 10:51:27 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 10:51:27.375631 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83dc82de-c95a-44ea-9c3e-6b1db818fbdb None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:51:27.398380 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:51:27.482960 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83dc82de-c95a-44ea-9c3e-6b1db818fbdb None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:51:27.530761 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aaa20bf-eb43-48f8-82f7-01fcaaca1e08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:51:27.531855 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aaa20bf-eb43-48f8-82f7-01fcaaca1e08 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:51:27.532135 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aaa20bf-eb43-48f8-82f7-01fcaaca1e08 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:51:27.532410 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aaa20bf-eb43-48f8-82f7-01fcaaca1e08 None admin] PATH_INFO: `/v3/users/00deb7d8c98747c6a68e18b5485c8a02` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:51:27.533749 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aaa20bf-eb43-48f8-82f7-01fcaaca1e08 None admin] RBAC: Authorizing `identity:get_user(user_id=00deb7d8c98747c6a68e18b5485c8a02)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:51:27.536101 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aaa20bf-eb43-48f8-82f7-01fcaaca1e08 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:51:27.538532 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 117/227] 10.222.0.40 () {56 vars in 1177 bytes} [Fri Aug 26 10:51:27 2022] GET /identity/v3/users/00deb7d8c98747c6a68e18b5485c8a02 => generated 283 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 10:51:27.545324 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aaa20bf-eb43-48f8-82f7-01fcaaca1e08 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:51:27.584165 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5757c24e-2787-464e-84cc-dd565859aede None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:51:27.660751 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd299bea-b9bc-435c-80be-40e908d73a94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:51:27.661458 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd299bea-b9bc-435c-80be-40e908d73a94 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:51:27.661734 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd299bea-b9bc-435c-80be-40e908d73a94 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:51:27.662574 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd299bea-b9bc-435c-80be-40e908d73a94 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:51:27.684500 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd299bea-b9bc-435c-80be-40e908d73a94 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 10:51:27.693071 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application [None req-cd299bea-b9bc-435c-80be-40e908d73a94 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 26 10:51:27.693071 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 10:51:27.693071 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 10:51:27.693071 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 10:51:27.693071 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 10:51:27.693071 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 10:51:27.693071 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 10:51:27.693071 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 10:51:27.693071 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 10:51:27.693071 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 10:51:27.693071 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 10:51:27.693071 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 10:51:27.693071 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 10:51:27.693071 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 10:51:27.693071 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 10:51:27.693071 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 10:51:27.693940 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 10:51:27.693940 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 10:51:27.693940 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 10:51:27.693940 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 10:51:27.693940 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 10:51:27.693940 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 10:51:27.693940 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 10:51:27.693940 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 10:51:27.693940 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 10:51:27.693940 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 10:51:27.693940 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 10:51:27.693940 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application with Lock( Aug 26 10:51:27.693940 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 10:51:27.693940 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._enter() Aug 26 10:51:27.693940 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 10:51:27.693940 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 10:51:27.694765 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 10:51:27.694765 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.creator() Aug 26 10:51:27.694765 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 10:51:27.694765 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application created_value = creator( Aug 26 10:51:27.694765 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 10:51:27.694765 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 10:51:27.694765 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 10:51:27.694765 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 10:51:27.694765 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 10:51:27.694765 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 10:51:27.694765 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 26 10:51:27.694765 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Aug 26 10:51:27.694765 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 111/228] 10.222.0.40 () {56 vars in 1114 bytes} [Fri Aug 26 10:51:27 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 337 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 10:51:27.702931 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd299bea-b9bc-435c-80be-40e908d73a94 None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:51:27.718390 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:51:27.742193 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aaa20bf-eb43-48f8-82f7-01fcaaca1e08 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:51:27.796010 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49598874-2e06-48a2-a362-52cbbc4ac1cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:51:27.797434 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49598874-2e06-48a2-a362-52cbbc4ac1cc None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:51:27.797909 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49598874-2e06-48a2-a362-52cbbc4ac1cc None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:51:27.798415 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49598874-2e06-48a2-a362-52cbbc4ac1cc None admin] PATH_INFO: `/v3/projects/15667f91e7f6473cb9aab41830d80f5e` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:51:27.799920 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49598874-2e06-48a2-a362-52cbbc4ac1cc None admin] RBAC: Authorizing `identity:get_project(project_id=15667f91e7f6473cb9aab41830d80f5e)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:51:27.801589 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49598874-2e06-48a2-a362-52cbbc4ac1cc None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:51:27.803583 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 118/229] 10.222.0.40 () {56 vars in 1186 bytes} [Fri Aug 26 10:51:27 2022] GET /identity/v3/projects/15667f91e7f6473cb9aab41830d80f5e => generated 302 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 10:51:27.814309 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:51:27.815933 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49598874-2e06-48a2-a362-52cbbc4ac1cc None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:51:27.910314 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd299bea-b9bc-435c-80be-40e908d73a94 None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:51:27.976211 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49598874-2e06-48a2-a362-52cbbc4ac1cc None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:51:27.978642 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f782f6-51c3-4841-a012-908e582a44f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:51:27.979266 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f782f6-51c3-4841-a012-908e582a44f5 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:51:27.979540 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f782f6-51c3-4841-a012-908e582a44f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:51:27.979859 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f782f6-51c3-4841-a012-908e582a44f5 None admin] PATH_INFO: `/v3/roles` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:51:27.980526 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f782f6-51c3-4841-a012-908e582a44f5 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 10:51:27.981062 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f782f6-51c3-4841-a012-908e582a44f5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:51:27.983610 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f782f6-51c3-4841-a012-908e582a44f5 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:51:27.999467 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 112/230] 10.222.0.40 () {56 vars in 1111 bytes} [Fri Aug 26 10:51:27 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 10:51:28.005460 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f782f6-51c3-4841-a012-908e582a44f5 None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:51:28.051626 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c55ed5-df16-4500-98c8-48494564c490 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:51:28.052412 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c55ed5-df16-4500-98c8-48494564c490 None admin] REQUEST_METHOD: `PUT` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:51:28.052688 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c55ed5-df16-4500-98c8-48494564c490 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:51:28.052962 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c55ed5-df16-4500-98c8-48494564c490 None admin] PATH_INFO: `/v3/projects/15667f91e7f6473cb9aab41830d80f5e/users/00deb7d8c98747c6a68e18b5485c8a02/roles/1f97e4602e2e4d6498913a81f9ae8d6a` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:51:28.054975 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c55ed5-df16-4500-98c8-48494564c490 None admin] RBAC: Authorizing `identity:create_grant(project_id=15667f91e7f6473cb9aab41830d80f5e, user_id=00deb7d8c98747c6a68e18b5485c8a02, role_id=1f97e4602e2e4d6498913a81f9ae8d6a)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:51:28.056438 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c55ed5-df16-4500-98c8-48494564c490 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:51:28.096580 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 119/231] 10.222.0.40 () {58 vars in 1439 bytes} [Fri Aug 26 10:51:27 2022] PUT /identity/v3/projects/15667f91e7f6473cb9aab41830d80f5e/users/00deb7d8c98747c6a68e18b5485c8a02/roles/1f97e4602e2e4d6498913a81f9ae8d6a => generated 0 bytes in 282 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 10:51:28.129561 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f782f6-51c3-4841-a012-908e582a44f5 None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:51:28.141153 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:51:28.186400 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb209b2c-0722-4eef-8688-9ceb1f5b78bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:51:28.187612 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb209b2c-0722-4eef-8688-9ceb1f5b78bd None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:51:28.187892 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb209b2c-0722-4eef-8688-9ceb1f5b78bd None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:51:28.188164 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb209b2c-0722-4eef-8688-9ceb1f5b78bd None admin] PATH_INFO: `/v3/users/00deb7d8c98747c6a68e18b5485c8a02` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:51:28.189344 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb209b2c-0722-4eef-8688-9ceb1f5b78bd None admin] RBAC: Authorizing `identity:get_user(user_id=00deb7d8c98747c6a68e18b5485c8a02)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:51:28.192208 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb209b2c-0722-4eef-8688-9ceb1f5b78bd None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:51:28.197949 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 113/232] 10.222.0.40 () {56 vars in 1177 bytes} [Fri Aug 26 10:51:28 2022] GET /identity/v3/users/00deb7d8c98747c6a68e18b5485c8a02 => generated 283 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 10:51:28.207214 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c55ed5-df16-4500-98c8-48494564c490 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:51:28.223918 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:51:28.342517 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c55ed5-df16-4500-98c8-48494564c490 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:51:28.379558 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edac7f12-ad72-4f6d-854f-7a0a5afc7dc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:51:28.380613 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edac7f12-ad72-4f6d-854f-7a0a5afc7dc7 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:51:28.381083 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edac7f12-ad72-4f6d-854f-7a0a5afc7dc7 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:51:28.381560 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edac7f12-ad72-4f6d-854f-7a0a5afc7dc7 None admin] PATH_INFO: `/v3/projects/15667f91e7f6473cb9aab41830d80f5e` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:51:28.383020 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edac7f12-ad72-4f6d-854f-7a0a5afc7dc7 None admin] RBAC: Authorizing `identity:get_project(project_id=15667f91e7f6473cb9aab41830d80f5e)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:51:28.384701 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edac7f12-ad72-4f6d-854f-7a0a5afc7dc7 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:51:28.386950 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 120/233] 10.222.0.40 () {56 vars in 1186 bytes} [Fri Aug 26 10:51:28 2022] GET /identity/v3/projects/15667f91e7f6473cb9aab41830d80f5e => generated 302 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 10:51:28.392314 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb209b2c-0722-4eef-8688-9ceb1f5b78bd None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:51:28.500392 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb209b2c-0722-4eef-8688-9ceb1f5b78bd None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:51:28.538488 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded0efc0-de75-4ca6-9045-5a79942007c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:51:28.539535 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded0efc0-de75-4ca6-9045-5a79942007c7 None admin] REQUEST_METHOD: `PUT` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:51:28.540018 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded0efc0-de75-4ca6-9045-5a79942007c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:51:28.540583 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded0efc0-de75-4ca6-9045-5a79942007c7 None admin] PATH_INFO: `/v3/projects/15667f91e7f6473cb9aab41830d80f5e/users/00deb7d8c98747c6a68e18b5485c8a02/roles/9c439f898126444f92abb01910f1d02b` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:51:28.543336 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ded0efc0-de75-4ca6-9045-5a79942007c7 None admin] RBAC: Authorizing `identity:create_grant(project_id=15667f91e7f6473cb9aab41830d80f5e, user_id=00deb7d8c98747c6a68e18b5485c8a02, role_id=9c439f898126444f92abb01910f1d02b)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:51:28.544993 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ded0efc0-de75-4ca6-9045-5a79942007c7 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:51:28.567403 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 114/234] 10.222.0.40 () {58 vars in 1439 bytes} [Fri Aug 26 10:51:28 2022] PUT /identity/v3/projects/15667f91e7f6473cb9aab41830d80f5e/users/00deb7d8c98747c6a68e18b5485c8a02/roles/9c439f898126444f92abb01910f1d02b => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 10:51:28.615227 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571ffd69-bca0-4880-ac97-aee307969b08 None None] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:51:28.615821 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571ffd69-bca0-4880-ac97-aee307969b08 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:51:28.616287 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571ffd69-bca0-4880-ac97-aee307969b08 None None] PATH_INFO: `/` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:51:28.617876 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 121/235] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 26 10:51:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 10:51:28.623846 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34dc6ed3-04d2-4cfb-8c8b-e27a9fe63051 None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:51:28.624474 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34dc6ed3-04d2-4cfb-8c8b-e27a9fe63051 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:51:28.624952 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34dc6ed3-04d2-4cfb-8c8b-e27a9fe63051 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:51:28.644749 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-34dc6ed3-04d2-4cfb-8c8b-e27a9fe63051 None None] MFA Rules not processed for user `ca155a48e2024c32af3fd2e19e5fac11`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 10:51:28.646350 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-34dc6ed3-04d2-4cfb-8c8b-e27a9fe63051 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 10:51:28.682491 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:51:28.729402 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 115/236] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 26 10:51:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 10:51:28.761126 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2b6892-cf4a-414f-bc5f-697052c57de1 None None] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:51:28.761446 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2b6892-cf4a-414f-bc5f-697052c57de1 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:51:28.761675 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2b6892-cf4a-414f-bc5f-697052c57de1 None None] PATH_INFO: `/` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:51:28.763340 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 122/237] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 26 10:51:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 10:51:28.768677 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb3e47f-0bef-420d-aee0-72bbff9c6b81 None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:51:28.768960 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb3e47f-0bef-420d-aee0-72bbff9c6b81 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:51:28.769233 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb3e47f-0bef-420d-aee0-72bbff9c6b81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:51:28.789231 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-4cb3e47f-0bef-420d-aee0-72bbff9c6b81 None None] MFA Rules not processed for user `ca155a48e2024c32af3fd2e19e5fac11`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 10:51:28.790453 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-4cb3e47f-0bef-420d-aee0-72bbff9c6b81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 10:51:28.881691 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 116/238] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 26 10:51:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 10:51:29.043034 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2b6892-cf4a-414f-bc5f-697052c57de1 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:51:29.132367 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:51:29.155566 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2b6892-cf4a-414f-bc5f-697052c57de1 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:51:29.184920 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb3e47f-0bef-420d-aee0-72bbff9c6b81 None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:51:29.192520 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981b87fb-21af-48db-99fe-776d4aa1a125 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:51:29.193168 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981b87fb-21af-48db-99fe-776d4aa1a125 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:51:29.193907 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981b87fb-21af-48db-99fe-776d4aa1a125 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:51:29.193907 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981b87fb-21af-48db-99fe-776d4aa1a125 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:51:29.195047 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-981b87fb-21af-48db-99fe-776d4aa1a125 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:51:29.196631 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-981b87fb-21af-48db-99fe-776d4aa1a125 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:51:29.198706 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 123/239] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Aug 26 10:51:29 2022] GET /identity/v3/domains/default => generated 199 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 10:51:29.203882 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981b87fb-21af-48db-99fe-776d4aa1a125 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:51:29.316498 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981b87fb-21af-48db-99fe-776d4aa1a125 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:51:29.359990 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b68e67-bddc-4660-8eee-2bd208f320c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:51:29.360599 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b68e67-bddc-4660-8eee-2bd208f320c1 None admin] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:51:29.360876 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b68e67-bddc-4660-8eee-2bd208f320c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:51:29.361147 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b68e67-bddc-4660-8eee-2bd208f320c1 None admin] PATH_INFO: `/v3/users` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:51:29.362405 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b68e67-bddc-4660-8eee-2bd208f320c1 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:51:29.362593 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb3e47f-0bef-420d-aee0-72bbff9c6b81 None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:51:29.364028 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b68e67-bddc-4660-8eee-2bd208f320c1 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:51:29.366278 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:51:29.388867 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 124/240] 10.222.0.40 () {60 vars in 1132 bytes} [Fri Aug 26 10:51:29 2022] POST /identity/v3/users => generated 293 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 10:51:29.394617 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e3a8d9-8798-41a0-b36e-cb7e9ef2b860 None None] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:51:29.394617 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e3a8d9-8798-41a0-b36e-cb7e9ef2b860 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:51:29.394617 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e3a8d9-8798-41a0-b36e-cb7e9ef2b860 None None] PATH_INFO: `/` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:51:29.394617 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 125/241] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 26 10:51:29 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 10:51:29.403065 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4e7725-9ed3-47e8-a170-34e1a8fa6e7f None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:51:29.403626 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4e7725-9ed3-47e8-a170-34e1a8fa6e7f None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:51:29.404042 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4e7725-9ed3-47e8-a170-34e1a8fa6e7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:51:29.416788 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:51:29.438494 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e88702d-8cef-4411-a346-66ac86d053e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:51:29.439152 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e88702d-8cef-4411-a346-66ac86d053e2 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:51:29.439429 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e88702d-8cef-4411-a346-66ac86d053e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:51:29.440227 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e88702d-8cef-4411-a346-66ac86d053e2 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:51:29.442166 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-af4e7725-9ed3-47e8-a170-34e1a8fa6e7f None None] MFA Rules not processed for user `ca155a48e2024c32af3fd2e19e5fac11`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 10:51:29.443681 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-af4e7725-9ed3-47e8-a170-34e1a8fa6e7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 10:51:29.471523 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e88702d-8cef-4411-a346-66ac86d053e2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 10:51:29.476481 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application [None req-8e88702d-8cef-4411-a346-66ac86d053e2 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 26 10:51:29.476481 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 10:51:29.476481 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 10:51:29.476481 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 10:51:29.476481 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 10:51:29.476481 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 10:51:29.476481 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 10:51:29.476481 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 10:51:29.476481 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 10:51:29.476481 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 10:51:29.476481 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 10:51:29.476481 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 10:51:29.476481 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 10:51:29.476481 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 10:51:29.476481 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 10:51:29.476481 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 10:51:29.477401 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 10:51:29.477401 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 10:51:29.477401 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 10:51:29.477401 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 10:51:29.477401 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 10:51:29.477401 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 10:51:29.477401 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 10:51:29.477401 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 10:51:29.477401 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 10:51:29.477401 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 10:51:29.477401 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 10:51:29.477401 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application with Lock( Aug 26 10:51:29.477401 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 10:51:29.477401 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._enter() Aug 26 10:51:29.477401 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 10:51:29.477401 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 10:51:29.478226 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 10:51:29.478226 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.creator() Aug 26 10:51:29.478226 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 10:51:29.478226 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application created_value = creator( Aug 26 10:51:29.478226 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 10:51:29.478226 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 10:51:29.478226 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 10:51:29.478226 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 10:51:29.478226 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 10:51:29.478226 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 10:51:29.478226 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 26 10:51:29.478226 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Aug 26 10:51:29.478226 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 117/242] 10.222.0.40 () {56 vars in 1099 bytes} [Fri Aug 26 10:51:29 2022] GET /identity/v3/roles/reader => generated 84 bytes in 293 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 26 10:51:29.484393 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e88702d-8cef-4411-a346-66ac86d053e2 None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:51:29.576285 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 126/243] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 26 10:51:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 10:51:29.648288 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e88702d-8cef-4411-a346-66ac86d053e2 None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:51:29.688394 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1087b760-573b-4432-ae75-dcd8bdbc47d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:51:29.689006 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1087b760-573b-4432-ae75-dcd8bdbc47d0 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:51:29.689283 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1087b760-573b-4432-ae75-dcd8bdbc47d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:51:29.689552 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1087b760-573b-4432-ae75-dcd8bdbc47d0 None admin] PATH_INFO: `/v3/roles` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:51:29.690840 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1087b760-573b-4432-ae75-dcd8bdbc47d0 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 10:51:29.691136 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1087b760-573b-4432-ae75-dcd8bdbc47d0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:51:29.692636 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1087b760-573b-4432-ae75-dcd8bdbc47d0 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:51:29.701768 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 118/244] 10.222.0.40 () {56 vars in 1101 bytes} [Fri Aug 26 10:51:29 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 10:51:29.707299 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4e7725-9ed3-47e8-a170-34e1a8fa6e7f None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:51:29.827731 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4e7725-9ed3-47e8-a170-34e1a8fa6e7f None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:51:29.862397 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:51:29.869639 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553d80b2-2bde-44e4-afad-2f08581a98e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:51:29.870359 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553d80b2-2bde-44e4-afad-2f08581a98e2 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:51:29.870638 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553d80b2-2bde-44e4-afad-2f08581a98e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:51:29.870906 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553d80b2-2bde-44e4-afad-2f08581a98e2 None admin] PATH_INFO: `/v3/users/455b47e8a24346cc8bb4b1ac0dcb75e3` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:51:29.886554 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-553d80b2-2bde-44e4-afad-2f08581a98e2 None admin] RBAC: Authorizing `identity:get_user(user_id=455b47e8a24346cc8bb4b1ac0dcb75e3)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:51:29.888080 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-553d80b2-2bde-44e4-afad-2f08581a98e2 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:51:29.889928 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 127/245] 10.222.0.40 () {56 vars in 1177 bytes} [Fri Aug 26 10:51:29 2022] GET /identity/v3/users/455b47e8a24346cc8bb4b1ac0dcb75e3 => generated 297 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 10:51:29.896001 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1087b760-573b-4432-ae75-dcd8bdbc47d0 None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:51:29.920970 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:51:29.961796 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553d80b2-2bde-44e4-afad-2f08581a98e2 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:51:30.029115 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1087b760-573b-4432-ae75-dcd8bdbc47d0 None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:51:30.067652 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3afba76-24ff-4fef-a883-1b3d129f25dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:51:30.068268 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3afba76-24ff-4fef-a883-1b3d129f25dd None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:51:30.068544 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3afba76-24ff-4fef-a883-1b3d129f25dd None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:51:30.068810 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3afba76-24ff-4fef-a883-1b3d129f25dd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:51:30.069439 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553d80b2-2bde-44e4-afad-2f08581a98e2 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:51:30.112115 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5bf54ff-4dfe-4504-a735-ab449c685b47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:51:30.113145 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bf54ff-4dfe-4504-a735-ab449c685b47 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:51:30.113780 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bf54ff-4dfe-4504-a735-ab449c685b47 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:51:30.114554 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bf54ff-4dfe-4504-a735-ab449c685b47 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:51:30.126144 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5bf54ff-4dfe-4504-a735-ab449c685b47 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 10:51:30.131911 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application [None req-e5bf54ff-4dfe-4504-a735-ab449c685b47 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 26 10:51:30.131911 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 10:51:30.131911 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 10:51:30.131911 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 10:51:30.131911 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 10:51:30.131911 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 10:51:30.131911 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 10:51:30.131911 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 10:51:30.131911 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 10:51:30.131911 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 10:51:30.131911 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 10:51:30.131911 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 10:51:30.131911 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 10:51:30.131911 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 10:51:30.131911 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 10:51:30.131911 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 10:51:30.132816 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 10:51:30.132816 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 10:51:30.132816 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 10:51:30.132816 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 10:51:30.132816 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 10:51:30.132816 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 10:51:30.132816 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 10:51:30.132816 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 10:51:30.132816 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 10:51:30.132816 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 10:51:30.132816 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 10:51:30.132816 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application with Lock( Aug 26 10:51:30.132816 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 10:51:30.132816 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._enter() Aug 26 10:51:30.132816 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 10:51:30.132816 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 10:51:30.133615 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 10:51:30.133615 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.creator() Aug 26 10:51:30.133615 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 10:51:30.133615 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application created_value = creator( Aug 26 10:51:30.133615 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 10:51:30.133615 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 10:51:30.133615 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 10:51:30.133615 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 10:51:30.133615 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 10:51:30.133615 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 10:51:30.133615 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 26 10:51:30.133615 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Aug 26 10:51:30.135846 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 128/246] 10.222.0.40 () {56 vars in 1099 bytes} [Fri Aug 26 10:51:29 2022] GET /identity/v3/roles/member => generated 84 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 26 10:51:30.143502 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5bf54ff-4dfe-4504-a735-ab449c685b47 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:51:30.154594 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3afba76-24ff-4fef-a883-1b3d129f25dd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:51:30.156613 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3afba76-24ff-4fef-a883-1b3d129f25dd None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:51:30.188180 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:51:30.207386 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:51:30.284986 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5bf54ff-4dfe-4504-a735-ab449c685b47 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:51:30.327203 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092fc1b0-e062-46e7-bee5-11b2adeee277 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:51:30.328202 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092fc1b0-e062-46e7-bee5-11b2adeee277 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:51:30.328711 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 119/247] 10.222.0.40 () {58 vars in 1282 bytes} [Fri Aug 26 10:51:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 433 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 26 10:51:30.328822 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092fc1b0-e062-46e7-bee5-11b2adeee277 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:51:30.329275 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092fc1b0-e062-46e7-bee5-11b2adeee277 None admin] PATH_INFO: `/v3/roles` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:51:30.330183 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-092fc1b0-e062-46e7-bee5-11b2adeee277 None admin] RBAC: Adding query filter params (name=member) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 10:51:30.331348 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-092fc1b0-e062-46e7-bee5-11b2adeee277 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:51:30.333074 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-092fc1b0-e062-46e7-bee5-11b2adeee277 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:51:30.334791 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3afba76-24ff-4fef-a883-1b3d129f25dd None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:51:30.344284 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 129/248] 10.222.0.40 () {56 vars in 1101 bytes} [Fri Aug 26 10:51:30 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 10:51:30.352406 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092fc1b0-e062-46e7-bee5-11b2adeee277 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:51:30.472346 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092fc1b0-e062-46e7-bee5-11b2adeee277 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:51:30.488961 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:51:30.515291 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d5580b-eb91-4ae3-8a5d-eca9622c2dee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:51:30.516338 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d5580b-eb91-4ae3-8a5d-eca9622c2dee None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:51:30.516840 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d5580b-eb91-4ae3-8a5d-eca9622c2dee None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:51:30.517305 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d5580b-eb91-4ae3-8a5d-eca9622c2dee None admin] PATH_INFO: `/v3/users/c59df4b4751540dc8126c66c4843c7a2` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:51:30.519544 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d5580b-eb91-4ae3-8a5d-eca9622c2dee None admin] RBAC: Authorizing `identity:get_user(user_id=c59df4b4751540dc8126c66c4843c7a2)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:51:30.521377 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d5580b-eb91-4ae3-8a5d-eca9622c2dee None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:51:30.523559 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 130/249] 10.222.0.40 () {56 vars in 1177 bytes} [Fri Aug 26 10:51:30 2022] GET /identity/v3/users/c59df4b4751540dc8126c66c4843c7a2 => generated 297 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 10:51:30.530105 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d5580b-eb91-4ae3-8a5d-eca9622c2dee None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:51:30.564096 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3afba76-24ff-4fef-a883-1b3d129f25dd None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:51:30.606049 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:51:30.649485 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d5580b-eb91-4ae3-8a5d-eca9622c2dee None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:51:30.654693 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae404f70-deec-44c4-9b73-079c9fd4959a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:51:30.655352 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae404f70-deec-44c4-9b73-079c9fd4959a None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:51:30.655633 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae404f70-deec-44c4-9b73-079c9fd4959a None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:51:30.655906 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae404f70-deec-44c4-9b73-079c9fd4959a None admin] PATH_INFO: `/v3/projects/15667f91e7f6473cb9aab41830d80f5e` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:51:30.657196 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae404f70-deec-44c4-9b73-079c9fd4959a None admin] RBAC: Authorizing `identity:get_project(project_id=15667f91e7f6473cb9aab41830d80f5e)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:51:30.659586 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae404f70-deec-44c4-9b73-079c9fd4959a None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:51:30.662746 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 120/250] 10.222.0.40 () {56 vars in 1186 bytes} [Fri Aug 26 10:51:30 2022] GET /identity/v3/projects/15667f91e7f6473cb9aab41830d80f5e => generated 302 bytes in 328 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 10:51:30.669700 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae404f70-deec-44c4-9b73-079c9fd4959a None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:51:30.691070 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a852bc-d20e-460b-b23e-5750b39f8fe2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:51:30.692448 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a852bc-d20e-460b-b23e-5750b39f8fe2 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:51:30.692949 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a852bc-d20e-460b-b23e-5750b39f8fe2 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:51:30.693448 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a852bc-d20e-460b-b23e-5750b39f8fe2 None admin] PATH_INFO: `/v3/projects/15667f91e7f6473cb9aab41830d80f5e` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:51:30.694999 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a852bc-d20e-460b-b23e-5750b39f8fe2 None admin] RBAC: Authorizing `identity:get_project(project_id=15667f91e7f6473cb9aab41830d80f5e)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:51:30.696647 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a852bc-d20e-460b-b23e-5750b39f8fe2 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:51:30.699040 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 131/251] 10.222.0.40 () {56 vars in 1186 bytes} [Fri Aug 26 10:51:30 2022] GET /identity/v3/projects/15667f91e7f6473cb9aab41830d80f5e => generated 302 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 10:51:30.705880 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a852bc-d20e-460b-b23e-5750b39f8fe2 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:51:30.763167 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:51:30.830981 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a852bc-d20e-460b-b23e-5750b39f8fe2 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:51:30.849515 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae404f70-deec-44c4-9b73-079c9fd4959a None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:51:30.880391 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4dd949b-85c0-4ba7-a513-1fbd20a4c524 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:51:30.881073 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4dd949b-85c0-4ba7-a513-1fbd20a4c524 None admin] REQUEST_METHOD: `PUT` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:51:30.881355 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4dd949b-85c0-4ba7-a513-1fbd20a4c524 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:51:30.881633 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4dd949b-85c0-4ba7-a513-1fbd20a4c524 None admin] PATH_INFO: `/v3/projects/15667f91e7f6473cb9aab41830d80f5e/users/c59df4b4751540dc8126c66c4843c7a2/roles/23e174ce743b46eabe618c212691da55` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:51:30.883647 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4dd949b-85c0-4ba7-a513-1fbd20a4c524 None admin] RBAC: Authorizing `identity:create_grant(project_id=15667f91e7f6473cb9aab41830d80f5e, user_id=c59df4b4751540dc8126c66c4843c7a2, role_id=23e174ce743b46eabe618c212691da55)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:51:30.885095 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4dd949b-85c0-4ba7-a513-1fbd20a4c524 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:51:30.896126 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94edec8f-541f-415a-9814-1cbe54c0ae18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:51:30.897711 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94edec8f-541f-415a-9814-1cbe54c0ae18 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:51:30.898432 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94edec8f-541f-415a-9814-1cbe54c0ae18 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:51:30.899035 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94edec8f-541f-415a-9814-1cbe54c0ae18 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:51:30.900516 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94edec8f-541f-415a-9814-1cbe54c0ae18 None admin] RBAC: Adding query filter params (role.id=dae11e55a3f343ca9954bc6a1e748eed, scope.project.id=15667f91e7f6473cb9aab41830d80f5e, user.id=455b47e8a24346cc8bb4b1ac0dcb75e3) {{(pid=81423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 10:51:30.901070 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94edec8f-541f-415a-9814-1cbe54c0ae18 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:51:30.901866 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 132/252] 10.222.0.40 () {58 vars in 1439 bytes} [Fri Aug 26 10:51:30 2022] PUT /identity/v3/projects/15667f91e7f6473cb9aab41830d80f5e/users/c59df4b4751540dc8126c66c4843c7a2/roles/23e174ce743b46eabe618c212691da55 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 26 10:51:30.903189 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94edec8f-541f-415a-9814-1cbe54c0ae18 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:51:30.910005 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5406fb40-8c89-4a32-96c0-5a1a94d64563 None None] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:51:30.910005 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5406fb40-8c89-4a32-96c0-5a1a94d64563 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:51:30.910005 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5406fb40-8c89-4a32-96c0-5a1a94d64563 None None] PATH_INFO: `/` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:51:30.910005 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 133/253] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 26 10:51:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 10:51:30.913526 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34c8a3e-6a47-4cfe-8384-cb266b846da1 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:51:30.913932 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34c8a3e-6a47-4cfe-8384-cb266b846da1 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:51:30.914245 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34c8a3e-6a47-4cfe-8384-cb266b846da1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:51:30.923286 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 121/254] 10.222.0.40 () {56 vars in 1374 bytes} [Fri Aug 26 10:51:30 2022] GET /identity/v3/role_assignments?user.id=455b47e8a24346cc8bb4b1ac0dcb75e3&scope.project.id=15667f91e7f6473cb9aab41830d80f5e&role.id=dae11e55a3f343ca9954bc6a1e748eed => generated 259 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 10:51:30.941173 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-b34c8a3e-6a47-4cfe-8384-cb266b846da1 None None] MFA Rules not processed for user `ca155a48e2024c32af3fd2e19e5fac11`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 10:51:30.942711 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-b34c8a3e-6a47-4cfe-8384-cb266b846da1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 10:51:31.047303 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 134/255] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 26 10:51:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 26 10:51:31.322565 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df21dd7-7f79-4be9-a30f-9d76777da584 None None] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:51:31.323037 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df21dd7-7f79-4be9-a30f-9d76777da584 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:51:31.323037 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df21dd7-7f79-4be9-a30f-9d76777da584 None None] PATH_INFO: `/` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:51:31.324378 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 122/256] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 26 10:51:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 10:51:31.329955 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70d4c49-f64d-4eae-84ff-a51f31a00291 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:51:31.331012 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70d4c49-f64d-4eae-84ff-a51f31a00291 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:51:31.331133 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70d4c49-f64d-4eae-84ff-a51f31a00291 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:51:31.351968 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-e70d4c49-f64d-4eae-84ff-a51f31a00291 None None] MFA Rules not processed for user `ca155a48e2024c32af3fd2e19e5fac11`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 10:51:31.353179 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-e70d4c49-f64d-4eae-84ff-a51f31a00291 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 10:51:31.387628 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:51:31.434533 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 135/257] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 26 10:51:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 10:51:31.460364 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df21dd7-7f79-4be9-a30f-9d76777da584 None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:51:31.532667 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:51:31.597169 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df21dd7-7f79-4be9-a30f-9d76777da584 None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:51:31.638683 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d765e63-a61b-4bb4-a0dd-becb3aa70e2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:51:31.639320 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d765e63-a61b-4bb4-a0dd-becb3aa70e2f None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:51:31.639602 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d765e63-a61b-4bb4-a0dd-becb3aa70e2f None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:51:31.639876 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d765e63-a61b-4bb4-a0dd-becb3aa70e2f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:51:31.651056 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d765e63-a61b-4bb4-a0dd-becb3aa70e2f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 10:51:31.655930 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application [None req-1d765e63-a61b-4bb4-a0dd-becb3aa70e2f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 26 10:51:31.655930 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 10:51:31.655930 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 10:51:31.655930 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 10:51:31.655930 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 10:51:31.655930 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 10:51:31.655930 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 10:51:31.655930 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 10:51:31.655930 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 10:51:31.655930 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 10:51:31.655930 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 10:51:31.655930 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 10:51:31.655930 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 10:51:31.655930 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 10:51:31.655930 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 10:51:31.655930 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 10:51:31.656795 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 10:51:31.656795 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 10:51:31.656795 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 10:51:31.656795 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 10:51:31.656795 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 10:51:31.656795 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 10:51:31.656795 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 10:51:31.656795 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 10:51:31.656795 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 10:51:31.656795 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 10:51:31.656795 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 10:51:31.656795 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application with Lock( Aug 26 10:51:31.656795 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 10:51:31.656795 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._enter() Aug 26 10:51:31.656795 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 10:51:31.656795 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 10:51:31.657574 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 10:51:31.657574 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.creator() Aug 26 10:51:31.657574 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 10:51:31.657574 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application created_value = creator( Aug 26 10:51:31.657574 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 10:51:31.657574 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 10:51:31.657574 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 10:51:31.657574 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 10:51:31.657574 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 10:51:31.657574 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 10:51:31.657574 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 26 10:51:31.657574 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Aug 26 10:51:31.657574 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 123/258] 10.222.0.40 () {56 vars in 1096 bytes} [Fri Aug 26 10:51:31 2022] GET /identity/v3/roles/admin => generated 83 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 10:51:31.663245 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70d4c49-f64d-4eae-84ff-a51f31a00291 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:51:31.733857 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d765e63-a61b-4bb4-a0dd-becb3aa70e2f None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:51:31.824949 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70d4c49-f64d-4eae-84ff-a51f31a00291 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:51:31.864658 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d765e63-a61b-4bb4-a0dd-becb3aa70e2f None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:51:31.888114 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:51:31.912822 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc202d1c-6f13-44c5-8a1c-579eb2d6a2cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:51:31.913454 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc202d1c-6f13-44c5-8a1c-579eb2d6a2cd None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:51:31.913726 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc202d1c-6f13-44c5-8a1c-579eb2d6a2cd None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:51:31.914004 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc202d1c-6f13-44c5-8a1c-579eb2d6a2cd None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:51:31.915181 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68eabd5-c0f4-460a-98ea-ac3213cf2f57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:51:31.915802 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68eabd5-c0f4-460a-98ea-ac3213cf2f57 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:51:31.916079 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68eabd5-c0f4-460a-98ea-ac3213cf2f57 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:51:31.916351 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68eabd5-c0f4-460a-98ea-ac3213cf2f57 None admin] PATH_INFO: `/v3/roles` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:51:31.917136 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d68eabd5-c0f4-460a-98ea-ac3213cf2f57 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 10:51:31.917440 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d68eabd5-c0f4-460a-98ea-ac3213cf2f57 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:51:31.918947 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d68eabd5-c0f4-460a-98ea-ac3213cf2f57 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:51:31.926604 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc202d1c-6f13-44c5-8a1c-579eb2d6a2cd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 10:51:31.932726 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application [None req-dc202d1c-6f13-44c5-8a1c-579eb2d6a2cd None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 26 10:51:31.932726 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 10:51:31.932726 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 10:51:31.932726 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 10:51:31.932726 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 10:51:31.932726 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 10:51:31.932726 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 10:51:31.932726 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 10:51:31.932726 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 10:51:31.932726 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 10:51:31.932726 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 10:51:31.932726 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 10:51:31.932726 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 10:51:31.932726 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 10:51:31.932726 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 10:51:31.932726 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 10:51:31.933799 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 10:51:31.933799 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 10:51:31.933799 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 10:51:31.933799 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 10:51:31.933799 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 10:51:31.933799 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 10:51:31.933799 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 10:51:31.933799 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 10:51:31.933799 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 10:51:31.933799 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 10:51:31.933799 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 10:51:31.933799 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application with Lock( Aug 26 10:51:31.933799 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 10:51:31.933799 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._enter() Aug 26 10:51:31.933799 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 10:51:31.933799 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 10:51:31.934885 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 10:51:31.934885 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.creator() Aug 26 10:51:31.934885 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 10:51:31.934885 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application created_value = creator( Aug 26 10:51:31.934885 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 10:51:31.934885 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 10:51:31.934885 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 10:51:31.934885 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 10:51:31.934885 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 10:51:31.934885 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 10:51:31.934885 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 26 10:51:31.934885 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Aug 26 10:51:31.937534 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 124/259] 10.222.0.40 () {56 vars in 1114 bytes} [Fri Aug 26 10:51:31 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 10:51:31.943660 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc202d1c-6f13-44c5-8a1c-579eb2d6a2cd None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:51:31.949296 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 136/260] 10.222.0.40 () {56 vars in 1099 bytes} [Fri Aug 26 10:51:31 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 10:51:31.954858 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68eabd5-c0f4-460a-98ea-ac3213cf2f57 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:51:32.024461 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:51:32.091976 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc202d1c-6f13-44c5-8a1c-579eb2d6a2cd None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:51:32.150884 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2936253a-3322-4162-8fa7-c9b420141a26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:51:32.151498 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2936253a-3322-4162-8fa7-c9b420141a26 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:51:32.151774 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2936253a-3322-4162-8fa7-c9b420141a26 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:51:32.152044 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2936253a-3322-4162-8fa7-c9b420141a26 None admin] PATH_INFO: `/v3/roles` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:51:32.152764 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2936253a-3322-4162-8fa7-c9b420141a26 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 10:51:32.153060 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2936253a-3322-4162-8fa7-c9b420141a26 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:51:32.154960 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2936253a-3322-4162-8fa7-c9b420141a26 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:51:32.172358 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 125/261] 10.222.0.40 () {56 vars in 1111 bytes} [Fri Aug 26 10:51:31 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 10:51:32.179349 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2936253a-3322-4162-8fa7-c9b420141a26 None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:51:32.214423 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68eabd5-c0f4-460a-98ea-ac3213cf2f57 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:51:32.221687 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:51:32.298681 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad363abe-c4ec-4a05-a5ef-720cb587a55c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:51:32.299502 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad363abe-c4ec-4a05-a5ef-720cb587a55c None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:51:32.299941 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad363abe-c4ec-4a05-a5ef-720cb587a55c None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:51:32.300221 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad363abe-c4ec-4a05-a5ef-720cb587a55c None admin] PATH_INFO: `/v3/users/00deb7d8c98747c6a68e18b5485c8a02` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:51:32.301428 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad363abe-c4ec-4a05-a5ef-720cb587a55c None admin] RBAC: Authorizing `identity:get_user(user_id=00deb7d8c98747c6a68e18b5485c8a02)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:51:32.306026 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad363abe-c4ec-4a05-a5ef-720cb587a55c None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:51:32.306149 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 137/262] 10.222.0.40 () {56 vars in 1177 bytes} [Fri Aug 26 10:51:31 2022] GET /identity/v3/users/00deb7d8c98747c6a68e18b5485c8a02 => generated 283 bytes in 351 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 10:51:32.308561 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2936253a-3322-4162-8fa7-c9b420141a26 None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:51:32.314328 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad363abe-c4ec-4a05-a5ef-720cb587a55c None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:51:32.379222 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4537251c-5492-42c3-a824-322405ec1588 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:51:32.379886 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4537251c-5492-42c3-a824-322405ec1588 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:51:32.380130 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4537251c-5492-42c3-a824-322405ec1588 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:51:32.380403 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4537251c-5492-42c3-a824-322405ec1588 None admin] PATH_INFO: `/v3/users/00deb7d8c98747c6a68e18b5485c8a02` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:51:32.383019 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4537251c-5492-42c3-a824-322405ec1588 None admin] RBAC: Authorizing `identity:get_user(user_id=00deb7d8c98747c6a68e18b5485c8a02)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:51:32.384468 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4537251c-5492-42c3-a824-322405ec1588 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:51:32.386119 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 126/263] 10.222.0.40 () {56 vars in 1177 bytes} [Fri Aug 26 10:51:32 2022] GET /identity/v3/users/00deb7d8c98747c6a68e18b5485c8a02 => generated 283 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 10:51:32.391746 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4537251c-5492-42c3-a824-322405ec1588 None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:51:32.485628 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:51:32.490943 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad363abe-c4ec-4a05-a5ef-720cb587a55c None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:51:32.552383 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335f137e-cc05-4087-98d0-d3ccb3d77cf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:51:32.553001 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335f137e-cc05-4087-98d0-d3ccb3d77cf5 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:51:32.553283 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335f137e-cc05-4087-98d0-d3ccb3d77cf5 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:51:32.553549 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335f137e-cc05-4087-98d0-d3ccb3d77cf5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:51:32.573159 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4537251c-5492-42c3-a824-322405ec1588 None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:51:32.577342 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:51:32.640030 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b3fc74-412f-4a64-90fb-f13124d0ebf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:51:32.640651 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b3fc74-412f-4a64-90fb-f13124d0ebf7 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:51:32.640929 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b3fc74-412f-4a64-90fb-f13124d0ebf7 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:51:32.641196 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b3fc74-412f-4a64-90fb-f13124d0ebf7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:51:32.682502 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-335f137e-cc05-4087-98d0-d3ccb3d77cf5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:51:32.683987 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-335f137e-cc05-4087-98d0-d3ccb3d77cf5 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:51:32.802153 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:51:32.805719 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b3fc74-412f-4a64-90fb-f13124d0ebf7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:51:32.807377 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b3fc74-412f-4a64-90fb-f13124d0ebf7 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:51:32.807865 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 138/264] 10.222.0.40 () {58 vars in 1282 bytes} [Fri Aug 26 10:51:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 494 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 26 10:51:32.815714 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335f137e-cc05-4087-98d0-d3ccb3d77cf5 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:51:32.916170 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 127/265] 10.222.0.40 () {58 vars in 1282 bytes} [Fri Aug 26 10:51:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 525 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 26 10:51:32.921677 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b3fc74-412f-4a64-90fb-f13124d0ebf7 None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:51:32.925112 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:51:32.967180 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335f137e-cc05-4087-98d0-d3ccb3d77cf5 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:51:33.015260 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02eaf5c9-f826-4d71-990f-239c6277cd7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:51:33.015917 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02eaf5c9-f826-4d71-990f-239c6277cd7e None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:51:33.016198 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02eaf5c9-f826-4d71-990f-239c6277cd7e None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:51:33.016478 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02eaf5c9-f826-4d71-990f-239c6277cd7e None admin] PATH_INFO: `/v3/projects/15667f91e7f6473cb9aab41830d80f5e` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:51:33.019537 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02eaf5c9-f826-4d71-990f-239c6277cd7e None admin] RBAC: Authorizing `identity:get_project(project_id=15667f91e7f6473cb9aab41830d80f5e)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:51:33.021006 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02eaf5c9-f826-4d71-990f-239c6277cd7e None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:51:33.022872 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 139/266] 10.222.0.40 () {56 vars in 1186 bytes} [Fri Aug 26 10:51:32 2022] GET /identity/v3/projects/15667f91e7f6473cb9aab41830d80f5e => generated 302 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 10:51:33.026165 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b3fc74-412f-4a64-90fb-f13124d0ebf7 None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:51:33.028391 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02eaf5c9-f826-4d71-990f-239c6277cd7e None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:51:33.062045 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e07291-53ff-4d8b-9506-214736bb8ff9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:51:33.062667 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e07291-53ff-4d8b-9506-214736bb8ff9 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:51:33.062956 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e07291-53ff-4d8b-9506-214736bb8ff9 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:51:33.063235 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e07291-53ff-4d8b-9506-214736bb8ff9 None admin] PATH_INFO: `/v3/projects/15667f91e7f6473cb9aab41830d80f5e` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:51:33.064399 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e07291-53ff-4d8b-9506-214736bb8ff9 None admin] RBAC: Authorizing `identity:get_project(project_id=15667f91e7f6473cb9aab41830d80f5e)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:51:33.065894 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e07291-53ff-4d8b-9506-214736bb8ff9 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:51:33.067586 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 128/267] 10.222.0.40 () {56 vars in 1186 bytes} [Fri Aug 26 10:51:32 2022] GET /identity/v3/projects/15667f91e7f6473cb9aab41830d80f5e => generated 302 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 10:51:33.073394 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e07291-53ff-4d8b-9506-214736bb8ff9 None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:51:33.135967 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:51:33.147641 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:51:33.187924 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e07291-53ff-4d8b-9506-214736bb8ff9 None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:51:33.191169 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02eaf5c9-f826-4d71-990f-239c6277cd7e None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:51:33.232591 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d887d5e0-6835-41c9-9dd6-7223467567f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:51:33.233796 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d887d5e0-6835-41c9-9dd6-7223467567f9 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:51:33.234282 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d887d5e0-6835-41c9-9dd6-7223467567f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:51:33.235155 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d887d5e0-6835-41c9-9dd6-7223467567f9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:51:33.236092 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d887d5e0-6835-41c9-9dd6-7223467567f9 None admin] RBAC: Adding query filter params (role.id=1f97e4602e2e4d6498913a81f9ae8d6a, scope.project.id=15667f91e7f6473cb9aab41830d80f5e, user.id=00deb7d8c98747c6a68e18b5485c8a02) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 10:51:33.237191 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d887d5e0-6835-41c9-9dd6-7223467567f9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:51:33.239630 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d887d5e0-6835-41c9-9dd6-7223467567f9 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:51:33.246382 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 140/268] 10.222.0.40 () {56 vars in 1374 bytes} [Fri Aug 26 10:51:33 2022] GET /identity/v3/role_assignments?user.id=00deb7d8c98747c6a68e18b5485c8a02&scope.project.id=15667f91e7f6473cb9aab41830d80f5e&role.id=1f97e4602e2e4d6498913a81f9ae8d6a => generated 608 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 10:51:33.247905 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709aa272-4dbe-423b-a958-00e62a1a422d None None] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:51:33.248646 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709aa272-4dbe-423b-a958-00e62a1a422d None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:51:33.249106 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709aa272-4dbe-423b-a958-00e62a1a422d None None] PATH_INFO: `/` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:51:33.250580 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 141/269] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 26 10:51:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 10:51:33.256296 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd215ba-3632-4d23-baf6-6058f4ae8b49 None None] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:51:33.257219 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd215ba-3632-4d23-baf6-6058f4ae8b49 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:51:33.257801 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd215ba-3632-4d23-baf6-6058f4ae8b49 None None] PATH_INFO: `/` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:51:33.259831 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 142/270] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 26 10:51:33 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 10:51:33.260333 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e32e99-e689-422e-8aeb-07f08988d9d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:51:33.260956 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e32e99-e689-422e-8aeb-07f08988d9d0 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:51:33.261232 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e32e99-e689-422e-8aeb-07f08988d9d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:51:33.261510 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e32e99-e689-422e-8aeb-07f08988d9d0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:51:33.262042 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c5f389-0ad7-451c-9370-ecbc10a23124 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:51:33.262508 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c5f389-0ad7-451c-9370-ecbc10a23124 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:51:33.262966 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c5f389-0ad7-451c-9370-ecbc10a23124 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:51:33.266361 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e32e99-e689-422e-8aeb-07f08988d9d0 None admin] RBAC: Adding query filter params (role.id=9c439f898126444f92abb01910f1d02b, scope.project.id=15667f91e7f6473cb9aab41830d80f5e, user.id=00deb7d8c98747c6a68e18b5485c8a02) {{(pid=81423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 10:51:33.266695 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e32e99-e689-422e-8aeb-07f08988d9d0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:51:33.268290 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e32e99-e689-422e-8aeb-07f08988d9d0 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:51:33.276939 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 129/271] 10.222.0.40 () {56 vars in 1374 bytes} [Fri Aug 26 10:51:33 2022] GET /identity/v3/role_assignments?user.id=00deb7d8c98747c6a68e18b5485c8a02&scope.project.id=15667f91e7f6473cb9aab41830d80f5e&role.id=9c439f898126444f92abb01910f1d02b => generated 608 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 10:51:33.278970 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76c222e-4406-44be-878d-7149ad6a6576 None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:51:33.279273 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76c222e-4406-44be-878d-7149ad6a6576 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:51:33.279548 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76c222e-4406-44be-878d-7149ad6a6576 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:51:33.286085 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-e4c5f389-0ad7-451c-9370-ecbc10a23124 None None] MFA Rules not processed for user `ca155a48e2024c32af3fd2e19e5fac11`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 10:51:33.287819 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-e4c5f389-0ad7-451c-9370-ecbc10a23124 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 10:51:33.304268 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-d76c222e-4406-44be-878d-7149ad6a6576 None None] MFA Rules not processed for user `ca155a48e2024c32af3fd2e19e5fac11`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 10:51:33.306357 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-d76c222e-4406-44be-878d-7149ad6a6576 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 10:51:33.385575 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 143/272] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 26 10:51:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 10:51:33.398605 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 130/273] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 26 10:51:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 10:51:33.723491 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c5f389-0ad7-451c-9370-ecbc10a23124 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:51:33.745231 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76c222e-4406-44be-878d-7149ad6a6576 None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:51:33.801495 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:51:33.828113 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:51:33.871505 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c5f389-0ad7-451c-9370-ecbc10a23124 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:51:33.889083 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76c222e-4406-44be-878d-7149ad6a6576 None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:51:33.920138 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f2f5fb-d30a-44b6-a4c5-ef9c2f6789e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:51:33.920878 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f2f5fb-d30a-44b6-a4c5-ef9c2f6789e6 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:51:33.921231 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f2f5fb-d30a-44b6-a4c5-ef9c2f6789e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:51:33.921579 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f2f5fb-d30a-44b6-a4c5-ef9c2f6789e6 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:51:33.935264 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f2f5fb-d30a-44b6-a4c5-ef9c2f6789e6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 10:51:33.935960 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996bbcdd-cffa-47a3-8487-b2447a988ca4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:51:33.936804 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996bbcdd-cffa-47a3-8487-b2447a988ca4 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:51:33.937244 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996bbcdd-cffa-47a3-8487-b2447a988ca4 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:51:33.937676 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996bbcdd-cffa-47a3-8487-b2447a988ca4 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:51:33.938969 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-996bbcdd-cffa-47a3-8487-b2447a988ca4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:51:33.941209 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application [None req-e6f2f5fb-d30a-44b6-a4c5-ef9c2f6789e6 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 26 10:51:33.941209 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 10:51:33.941209 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 10:51:33.941209 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 10:51:33.941209 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 10:51:33.941209 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 10:51:33.941209 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 10:51:33.941209 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 10:51:33.941209 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 10:51:33.941209 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 10:51:33.941209 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 10:51:33.941209 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 10:51:33.941209 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 10:51:33.941209 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 10:51:33.941209 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 10:51:33.941209 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 10:51:33.942107 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 10:51:33.942107 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 10:51:33.942107 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 10:51:33.942107 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 10:51:33.942107 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 10:51:33.942107 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 10:51:33.942107 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 10:51:33.942107 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 10:51:33.942107 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 10:51:33.942107 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 10:51:33.942107 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 10:51:33.942107 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application with Lock( Aug 26 10:51:33.942107 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 10:51:33.942107 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._enter() Aug 26 10:51:33.942107 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 10:51:33.942107 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 10:51:33.942908 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 10:51:33.942908 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.creator() Aug 26 10:51:33.942908 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 10:51:33.942908 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application created_value = creator( Aug 26 10:51:33.942908 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 10:51:33.942908 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 10:51:33.942908 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 10:51:33.942908 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 10:51:33.942908 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 10:51:33.942908 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 10:51:33.942908 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 26 10:51:33.942908 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Aug 26 10:51:33.942908 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 144/274] 10.222.0.40 () {56 vars in 1099 bytes} [Fri Aug 26 10:51:33 2022] GET /identity/v3/roles/member => generated 84 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 10:51:33.944506 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-996bbcdd-cffa-47a3-8487-b2447a988ca4 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:51:33.946366 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 131/275] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Aug 26 10:51:33 2022] GET /identity/v3/domains/default => generated 199 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 10:51:33.948034 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f2f5fb-d30a-44b6-a4c5-ef9c2f6789e6 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:51:33.951783 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996bbcdd-cffa-47a3-8487-b2447a988ca4 None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:51:34.057512 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996bbcdd-cffa-47a3-8487-b2447a988ca4 None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:51:34.059157 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f2f5fb-d30a-44b6-a4c5-ef9c2f6789e6 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:51:34.082392 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:51:34.090681 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:51:34.097389 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2f5030-0b41-4c86-acee-854ee4bb9daa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:51:34.098248 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2f5030-0b41-4c86-acee-854ee4bb9daa None admin] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:51:34.098673 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2f5030-0b41-4c86-acee-854ee4bb9daa None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:51:34.098787 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6730f06-4f01-4206-9ca9-85b843dca92f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:51:34.099190 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2f5030-0b41-4c86-acee-854ee4bb9daa None admin] PATH_INFO: `/v3/users` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:51:34.099316 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6730f06-4f01-4206-9ca9-85b843dca92f None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:51:34.099569 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6730f06-4f01-4206-9ca9-85b843dca92f None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:51:34.099839 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6730f06-4f01-4206-9ca9-85b843dca92f None admin] PATH_INFO: `/v3/roles` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:51:34.100385 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd2f5030-0b41-4c86-acee-854ee4bb9daa None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:51:34.100494 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6730f06-4f01-4206-9ca9-85b843dca92f None admin] RBAC: Adding query filter params (name=member) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 10:51:34.100777 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6730f06-4f01-4206-9ca9-85b843dca92f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:51:34.102171 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6730f06-4f01-4206-9ca9-85b843dca92f None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:51:34.102463 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd2f5030-0b41-4c86-acee-854ee4bb9daa None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:51:34.115275 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 145/276] 10.222.0.40 () {56 vars in 1101 bytes} [Fri Aug 26 10:51:33 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 10:51:34.115275 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f956e1-9fad-4eb1-ae51-42cfd11966bb None None] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:51:34.115275 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f956e1-9fad-4eb1-ae51-42cfd11966bb None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:51:34.115275 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f956e1-9fad-4eb1-ae51-42cfd11966bb None None] PATH_INFO: `/` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:51:34.115275 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 146/277] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 26 10:51:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 10:51:34.119394 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a86040-4de5-49d4-9f14-25bd3bc5c4d9 None None] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:51:34.119724 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a86040-4de5-49d4-9f14-25bd3bc5c4d9 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:51:34.120135 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a86040-4de5-49d4-9f14-25bd3bc5c4d9 None None] PATH_INFO: `/` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:51:34.121749 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 147/278] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 26 10:51:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 10:51:34.122492 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a86040-4de5-49d4-9f14-25bd3bc5c4d9 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:51:34.142008 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 132/279] 10.222.0.40 () {60 vars in 1132 bytes} [Fri Aug 26 10:51:33 2022] POST /identity/v3/users => generated 293 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 26 10:51:34.142008 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a85a5d-6ecc-44a9-9df8-459ea6165a89 None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:51:34.142008 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a85a5d-6ecc-44a9-9df8-459ea6165a89 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:51:34.142008 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a85a5d-6ecc-44a9-9df8-459ea6165a89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:51:34.155219 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-b4a85a5d-6ecc-44a9-9df8-459ea6165a89 None None] MFA Rules not processed for user `ca155a48e2024c32af3fd2e19e5fac11`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 10:51:34.156465 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-b4a85a5d-6ecc-44a9-9df8-459ea6165a89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 10:51:34.237775 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 133/280] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 26 10:51:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 10:51:34.239098 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926d9300-0282-4169-a5f9-b2eebeee10e8 None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:51:34.239387 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926d9300-0282-4169-a5f9-b2eebeee10e8 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:51:34.239659 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926d9300-0282-4169-a5f9-b2eebeee10e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:51:34.264623 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-926d9300-0282-4169-a5f9-b2eebeee10e8 None None] MFA Rules not processed for user `ca155a48e2024c32af3fd2e19e5fac11`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 10:51:34.265839 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-926d9300-0282-4169-a5f9-b2eebeee10e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 10:51:34.282478 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a86040-4de5-49d4-9f14-25bd3bc5c4d9 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:51:34.332337 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5bce90-af16-4d6e-9818-176131cbbc8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:51:34.332337 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5bce90-af16-4d6e-9818-176131cbbc8b None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:51:34.332337 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5bce90-af16-4d6e-9818-176131cbbc8b None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:51:34.332337 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5bce90-af16-4d6e-9818-176131cbbc8b None admin] PATH_INFO: `/v3/users/981035b471a54d77800a40267a6a30ef` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:51:34.347476 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c5bce90-af16-4d6e-9818-176131cbbc8b None admin] RBAC: Authorizing `identity:get_user(user_id=981035b471a54d77800a40267a6a30ef)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:51:34.348969 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c5bce90-af16-4d6e-9818-176131cbbc8b None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:51:34.351191 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 148/281] 10.222.0.40 () {56 vars in 1177 bytes} [Fri Aug 26 10:51:34 2022] GET /identity/v3/users/981035b471a54d77800a40267a6a30ef => generated 293 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 10:51:34.356872 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5bce90-af16-4d6e-9818-176131cbbc8b None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:51:34.364472 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 134/282] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 26 10:51:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 10:51:34.414351 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:51:34.486251 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5bce90-af16-4d6e-9818-176131cbbc8b None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:51:34.524405 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e32651-e8f5-4689-8e02-8c2f2535926e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:51:34.525370 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e32651-e8f5-4689-8e02-8c2f2535926e None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:51:34.525839 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e32651-e8f5-4689-8e02-8c2f2535926e None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:51:34.526338 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e32651-e8f5-4689-8e02-8c2f2535926e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:51:34.527248 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e32651-e8f5-4689-8e02-8c2f2535926e None admin] RBAC: Adding query filter params (role.id=23e174ce743b46eabe618c212691da55, user.id=981035b471a54d77800a40267a6a30ef, scope.system=all) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 10:51:34.527777 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e32651-e8f5-4689-8e02-8c2f2535926e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:51:34.529631 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e32651-e8f5-4689-8e02-8c2f2535926e None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:51:34.536197 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 149/283] 10.222.0.40 () {56 vars in 1308 bytes} [Fri Aug 26 10:51:34 2022] GET /identity/v3/role_assignments?user.id=981035b471a54d77800a40267a6a30ef&scope.system=all&role.id=23e174ce743b46eabe618c212691da55 => generated 226 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 10:51:34.589036 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926d9300-0282-4169-a5f9-b2eebeee10e8 None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:51:34.656718 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:51:34.686291 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e32651-e8f5-4689-8e02-8c2f2535926e None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:51:34.721069 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926d9300-0282-4169-a5f9-b2eebeee10e8 None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:51:34.760308 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05aed034-a0f4-4fa6-920b-fdae98fab647 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:51:34.761020 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05aed034-a0f4-4fa6-920b-fdae98fab647 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:51:34.761299 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05aed034-a0f4-4fa6-920b-fdae98fab647 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:51:34.761563 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05aed034-a0f4-4fa6-920b-fdae98fab647 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:51:34.774170 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05aed034-a0f4-4fa6-920b-fdae98fab647 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 10:51:34.779675 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application [None req-05aed034-a0f4-4fa6-920b-fdae98fab647 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 26 10:51:34.779675 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 10:51:34.779675 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 10:51:34.779675 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 10:51:34.779675 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 10:51:34.779675 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 10:51:34.779675 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 10:51:34.779675 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 10:51:34.779675 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 10:51:34.779675 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 10:51:34.779675 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 10:51:34.779675 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 10:51:34.779675 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 10:51:34.779675 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 10:51:34.779675 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 10:51:34.779675 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 10:51:34.780557 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 10:51:34.780557 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 10:51:34.780557 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 10:51:34.780557 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 10:51:34.780557 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 10:51:34.780557 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 10:51:34.780557 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 10:51:34.780557 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 10:51:34.780557 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 10:51:34.780557 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 10:51:34.780557 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 10:51:34.780557 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application with Lock( Aug 26 10:51:34.780557 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 10:51:34.780557 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._enter() Aug 26 10:51:34.780557 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 10:51:34.780557 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 10:51:34.781325 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 10:51:34.781325 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.creator() Aug 26 10:51:34.781325 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 10:51:34.781325 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application created_value = creator( Aug 26 10:51:34.781325 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 10:51:34.781325 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 10:51:34.781325 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 10:51:34.781325 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 10:51:34.781325 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 10:51:34.781325 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 10:51:34.781325 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 26 10:51:34.781325 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Aug 26 10:51:34.781325 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 135/284] 10.222.0.40 () {56 vars in 1099 bytes} [Fri Aug 26 10:51:34 2022] GET /identity/v3/roles/member => generated 84 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 10:51:34.794372 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05aed034-a0f4-4fa6-920b-fdae98fab647 None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:51:34.840988 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e32651-e8f5-4689-8e02-8c2f2535926e None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:51:34.862761 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:51:34.922244 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0709444-5b0f-49af-b4c6-8d9fac282b9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:51:34.922876 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0709444-5b0f-49af-b4c6-8d9fac282b9f None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:51:34.923149 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0709444-5b0f-49af-b4c6-8d9fac282b9f None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:51:34.923418 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0709444-5b0f-49af-b4c6-8d9fac282b9f None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:51:34.950952 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0709444-5b0f-49af-b4c6-8d9fac282b9f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 10:51:34.957727 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application [None req-a0709444-5b0f-49af-b4c6-8d9fac282b9f None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 26 10:51:34.957727 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 10:51:34.957727 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 10:51:34.957727 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 10:51:34.957727 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 10:51:34.957727 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 10:51:34.957727 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 10:51:34.957727 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 10:51:34.957727 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 10:51:34.957727 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 10:51:34.957727 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 10:51:34.957727 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 10:51:34.957727 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 10:51:34.957727 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 10:51:34.957727 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 10:51:34.957727 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 10:51:34.958666 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 10:51:34.958666 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 10:51:34.958666 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 10:51:34.958666 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 10:51:34.958666 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 10:51:34.958666 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 10:51:34.958666 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 10:51:34.958666 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 10:51:34.958666 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 10:51:34.958666 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 10:51:34.958666 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 10:51:34.958666 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application with Lock( Aug 26 10:51:34.958666 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 10:51:34.958666 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._enter() Aug 26 10:51:34.958666 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 10:51:34.958666 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 10:51:34.959450 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 10:51:34.959450 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.creator() Aug 26 10:51:34.959450 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 10:51:34.959450 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application created_value = creator( Aug 26 10:51:34.959450 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 10:51:34.959450 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 10:51:34.959450 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 10:51:34.959450 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 10:51:34.959450 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 10:51:34.959450 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 10:51:34.959450 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 26 10:51:34.959450 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Aug 26 10:51:34.962406 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 150/285] 10.222.0.40 () {56 vars in 1099 bytes} [Fri Aug 26 10:51:34 2022] GET /identity/v3/roles/reader => generated 84 bytes in 276 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 10:51:34.971494 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0709444-5b0f-49af-b4c6-8d9fac282b9f None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:51:34.994761 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05aed034-a0f4-4fa6-920b-fdae98fab647 None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:51:35.010841 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:51:35.063919 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d5ef8c-5bc4-41b2-98e0-fdcbc813ccfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:51:35.065438 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d5ef8c-5bc4-41b2-98e0-fdcbc813ccfa None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:51:35.065822 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d5ef8c-5bc4-41b2-98e0-fdcbc813ccfa None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:51:35.067349 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d5ef8c-5bc4-41b2-98e0-fdcbc813ccfa None admin] PATH_INFO: `/v3/roles` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:51:35.067349 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20d5ef8c-5bc4-41b2-98e0-fdcbc813ccfa None admin] RBAC: Adding query filter params (name=member) {{(pid=81423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 10:51:35.067349 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20d5ef8c-5bc4-41b2-98e0-fdcbc813ccfa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:51:35.069278 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20d5ef8c-5bc4-41b2-98e0-fdcbc813ccfa None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:51:35.078133 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 136/286] 10.222.0.40 () {56 vars in 1101 bytes} [Fri Aug 26 10:51:34 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 10:51:35.087283 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d5ef8c-5bc4-41b2-98e0-fdcbc813ccfa None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:51:35.153604 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0709444-5b0f-49af-b4c6-8d9fac282b9f None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:51:35.218108 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6189df-b484-44ce-88f7-66ba3de1296c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:51:35.220034 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6189df-b484-44ce-88f7-66ba3de1296c None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:51:35.220345 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6189df-b484-44ce-88f7-66ba3de1296c None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:51:35.220616 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6189df-b484-44ce-88f7-66ba3de1296c None admin] PATH_INFO: `/v3/roles` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:51:35.226119 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba6189df-b484-44ce-88f7-66ba3de1296c None admin] RBAC: Adding query filter params (name=reader) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 10:51:35.226497 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba6189df-b484-44ce-88f7-66ba3de1296c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:51:35.227928 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba6189df-b484-44ce-88f7-66ba3de1296c None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:51:35.243950 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 151/287] 10.222.0.40 () {56 vars in 1101 bytes} [Fri Aug 26 10:51:34 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 10:51:35.251319 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6189df-b484-44ce-88f7-66ba3de1296c None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:51:35.284670 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d5ef8c-5bc4-41b2-98e0-fdcbc813ccfa None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:51:35.304519 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:51:35.352314 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe8b8fe-329d-4a79-867d-036f6aef1bba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:51:35.352951 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe8b8fe-329d-4a79-867d-036f6aef1bba None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:51:35.353226 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe8b8fe-329d-4a79-867d-036f6aef1bba None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:51:35.353489 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe8b8fe-329d-4a79-867d-036f6aef1bba None admin] PATH_INFO: `/v3/users/c59df4b4751540dc8126c66c4843c7a2` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:51:35.355470 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe8b8fe-329d-4a79-867d-036f6aef1bba None admin] RBAC: Authorizing `identity:get_user(user_id=c59df4b4751540dc8126c66c4843c7a2)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:51:35.356901 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe8b8fe-329d-4a79-867d-036f6aef1bba None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:51:35.362553 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 137/288] 10.222.0.40 () {56 vars in 1177 bytes} [Fri Aug 26 10:51:35 2022] GET /identity/v3/users/c59df4b4751540dc8126c66c4843c7a2 => generated 297 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 10:51:35.369590 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe8b8fe-329d-4a79-867d-036f6aef1bba None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:51:35.442283 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6189df-b484-44ce-88f7-66ba3de1296c None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:51:35.446317 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:51:35.509727 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50660a2-b596-4319-b485-37b389ea6a26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:51:35.510613 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50660a2-b596-4319-b485-37b389ea6a26 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:51:35.510885 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50660a2-b596-4319-b485-37b389ea6a26 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:51:35.511151 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50660a2-b596-4319-b485-37b389ea6a26 None admin] PATH_INFO: `/v3/users/455b47e8a24346cc8bb4b1ac0dcb75e3` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:51:35.516485 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b50660a2-b596-4319-b485-37b389ea6a26 None admin] RBAC: Authorizing `identity:get_user(user_id=455b47e8a24346cc8bb4b1ac0dcb75e3)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:51:35.518774 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b50660a2-b596-4319-b485-37b389ea6a26 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:51:35.520408 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 152/289] 10.222.0.40 () {56 vars in 1177 bytes} [Fri Aug 26 10:51:35 2022] GET /identity/v3/users/455b47e8a24346cc8bb4b1ac0dcb75e3 => generated 297 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 10:51:35.527744 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50660a2-b596-4319-b485-37b389ea6a26 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:51:35.579151 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe8b8fe-329d-4a79-867d-036f6aef1bba None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:51:35.615741 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2261620a-3aa7-4fa6-9855-595ecce78b49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:51:35.616349 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2261620a-3aa7-4fa6-9855-595ecce78b49 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:51:35.616615 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2261620a-3aa7-4fa6-9855-595ecce78b49 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:51:35.616884 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2261620a-3aa7-4fa6-9855-595ecce78b49 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:51:35.670848 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:51:35.681876 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50660a2-b596-4319-b485-37b389ea6a26 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:51:35.687096 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2261620a-3aa7-4fa6-9855-595ecce78b49 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:51:35.688578 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2261620a-3aa7-4fa6-9855-595ecce78b49 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:51:35.729989 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:51:35.733203 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9edef30b-449e-4df0-9566-2fa3899ca8fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:51:35.733848 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9edef30b-449e-4df0-9566-2fa3899ca8fe None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:51:35.734219 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9edef30b-449e-4df0-9566-2fa3899ca8fe None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:51:35.734498 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9edef30b-449e-4df0-9566-2fa3899ca8fe None admin] PATH_INFO: `/v3/projects/15667f91e7f6473cb9aab41830d80f5e` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:51:35.738128 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9edef30b-449e-4df0-9566-2fa3899ca8fe None admin] RBAC: Authorizing `identity:get_project(project_id=15667f91e7f6473cb9aab41830d80f5e)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:51:35.740760 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9edef30b-449e-4df0-9566-2fa3899ca8fe None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:51:35.742655 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 153/290] 10.222.0.40 () {56 vars in 1186 bytes} [Fri Aug 26 10:51:35 2022] GET /identity/v3/projects/15667f91e7f6473cb9aab41830d80f5e => generated 302 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 10:51:35.751301 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9edef30b-449e-4df0-9566-2fa3899ca8fe None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:51:35.795732 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 138/291] 10.222.0.40 () {58 vars in 1282 bytes} [Fri Aug 26 10:51:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 430 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 26 10:51:35.801733 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2261620a-3aa7-4fa6-9855-595ecce78b49 None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:51:35.923907 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2261620a-3aa7-4fa6-9855-595ecce78b49 None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:51:35.979293 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9edef30b-449e-4df0-9566-2fa3899ca8fe None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:51:36.017247 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4375ea5-ecca-434e-a94d-7cb5be33408a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:51:36.018821 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4375ea5-ecca-434e-a94d-7cb5be33408a None admin] REQUEST_METHOD: `PUT` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:51:36.019106 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4375ea5-ecca-434e-a94d-7cb5be33408a None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:51:36.019378 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4375ea5-ecca-434e-a94d-7cb5be33408a None admin] PATH_INFO: `/v3/projects/15667f91e7f6473cb9aab41830d80f5e/users/455b47e8a24346cc8bb4b1ac0dcb75e3/roles/dae11e55a3f343ca9954bc6a1e748eed` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:51:36.021246 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a8495c-d25e-4f0a-a4e6-b62cccf44f11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:51:36.021877 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a8495c-d25e-4f0a-a4e6-b62cccf44f11 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:51:36.022172 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a8495c-d25e-4f0a-a4e6-b62cccf44f11 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:51:36.022438 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a8495c-d25e-4f0a-a4e6-b62cccf44f11 None admin] PATH_INFO: `/v3/projects/15667f91e7f6473cb9aab41830d80f5e` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:51:36.023549 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a8495c-d25e-4f0a-a4e6-b62cccf44f11 None admin] RBAC: Authorizing `identity:get_project(project_id=15667f91e7f6473cb9aab41830d80f5e)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:51:36.024980 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a8495c-d25e-4f0a-a4e6-b62cccf44f11 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:51:36.026932 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 139/292] 10.222.0.40 () {56 vars in 1186 bytes} [Fri Aug 26 10:51:35 2022] GET /identity/v3/projects/15667f91e7f6473cb9aab41830d80f5e => generated 302 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 10:51:36.032402 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4375ea5-ecca-434e-a94d-7cb5be33408a None admin] RBAC: Authorizing `identity:create_grant(project_id=15667f91e7f6473cb9aab41830d80f5e, user_id=455b47e8a24346cc8bb4b1ac0dcb75e3, role_id=dae11e55a3f343ca9954bc6a1e748eed)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:51:36.033083 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a8495c-d25e-4f0a-a4e6-b62cccf44f11 None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:51:36.038014 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4375ea5-ecca-434e-a94d-7cb5be33408a None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:51:36.048033 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:51:36.061518 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 154/293] 10.222.0.40 () {58 vars in 1439 bytes} [Fri Aug 26 10:51:35 2022] PUT /identity/v3/projects/15667f91e7f6473cb9aab41830d80f5e/users/455b47e8a24346cc8bb4b1ac0dcb75e3/roles/dae11e55a3f343ca9954bc6a1e748eed => generated 0 bytes in 311 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 10:51:36.168635 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a8495c-d25e-4f0a-a4e6-b62cccf44f11 None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:51:36.218657 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042d6b3a-9c37-424b-bcc3-595b61e75c3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:51:36.219637 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042d6b3a-9c37-424b-bcc3-595b61e75c3c None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:51:36.220135 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042d6b3a-9c37-424b-bcc3-595b61e75c3c None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:51:36.220629 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042d6b3a-9c37-424b-bcc3-595b61e75c3c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:51:36.222372 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-042d6b3a-9c37-424b-bcc3-595b61e75c3c None admin] RBAC: Adding query filter params (role.id=23e174ce743b46eabe618c212691da55, scope.project.id=15667f91e7f6473cb9aab41830d80f5e, user.id=c59df4b4751540dc8126c66c4843c7a2) {{(pid=81423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 10:51:36.222949 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-042d6b3a-9c37-424b-bcc3-595b61e75c3c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:51:36.224666 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-042d6b3a-9c37-424b-bcc3-595b61e75c3c None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:51:36.235405 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 140/294] 10.222.0.40 () {56 vars in 1374 bytes} [Fri Aug 26 10:51:36 2022] GET /identity/v3/role_assignments?user.id=c59df4b4751540dc8126c66c4843c7a2&scope.project.id=15667f91e7f6473cb9aab41830d80f5e&role.id=23e174ce743b46eabe618c212691da55 => generated 608 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 10:51:36.663226 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48704add-5d84-4991-941f-d0c4607bc86f None None] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:51:36.663920 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48704add-5d84-4991-941f-d0c4607bc86f None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:51:36.664400 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48704add-5d84-4991-941f-d0c4607bc86f None None] PATH_INFO: `/` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:51:36.666263 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 155/295] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 26 10:51:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 10:51:36.671139 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b1b9b4-d4f8-46f8-8ec6-f1c588a9b8a7 None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:51:36.671984 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b1b9b4-d4f8-46f8-8ec6-f1c588a9b8a7 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:51:36.672455 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b1b9b4-d4f8-46f8-8ec6-f1c588a9b8a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:51:36.695873 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-d7b1b9b4-d4f8-46f8-8ec6-f1c588a9b8a7 None None] MFA Rules not processed for user `ca155a48e2024c32af3fd2e19e5fac11`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 10:51:36.697420 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-d7b1b9b4-d4f8-46f8-8ec6-f1c588a9b8a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 10:51:36.780000 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:51:36.787921 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 141/296] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 26 10:51:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 10:51:36.880469 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f473a28-d5a4-45d2-ac38-d4b706ed82a2 None None] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:51:36.881014 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f473a28-d5a4-45d2-ac38-d4b706ed82a2 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:51:36.881489 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f473a28-d5a4-45d2-ac38-d4b706ed82a2 None None] PATH_INFO: `/` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:51:36.883412 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 156/297] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 26 10:51:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 10:51:36.888925 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d111ee7-bc5f-4ae6-b2a8-2a597cc771b7 None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:51:36.889210 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d111ee7-bc5f-4ae6-b2a8-2a597cc771b7 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:51:36.889481 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d111ee7-bc5f-4ae6-b2a8-2a597cc771b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:51:36.909508 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-8d111ee7-bc5f-4ae6-b2a8-2a597cc771b7 None None] MFA Rules not processed for user `ca155a48e2024c32af3fd2e19e5fac11`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 10:51:36.911552 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-8d111ee7-bc5f-4ae6-b2a8-2a597cc771b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 10:51:36.918853 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482645c1-bac7-4d72-8ed8-932327f15c3b None None] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:51:36.919404 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482645c1-bac7-4d72-8ed8-932327f15c3b None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:51:36.919942 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482645c1-bac7-4d72-8ed8-932327f15c3b None None] PATH_INFO: `/` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:51:36.921744 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 157/298] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 26 10:51:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 10:51:36.929569 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321ed80c-6ee0-49f2-8b05-439f5811b2e0 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:51:36.929949 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321ed80c-6ee0-49f2-8b05-439f5811b2e0 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:51:36.930339 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321ed80c-6ee0-49f2-8b05-439f5811b2e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:51:36.951264 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-321ed80c-6ee0-49f2-8b05-439f5811b2e0 None None] MFA Rules not processed for user `ca155a48e2024c32af3fd2e19e5fac11`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 10:51:36.952617 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-321ed80c-6ee0-49f2-8b05-439f5811b2e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 10:51:36.974340 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:51:37.000890 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 142/299] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 26 10:51:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 10:51:37.037301 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 158/300] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 26 10:51:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 26 10:51:37.064210 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d111ee7-bc5f-4ae6-b2a8-2a597cc771b7 None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:51:37.169666 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d111ee7-bc5f-4ae6-b2a8-2a597cc771b7 None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:51:37.199050 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:51:37.206201 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71446744-3dd9-4599-8849-9b0e9060f417 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:51:37.206834 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71446744-3dd9-4599-8849-9b0e9060f417 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:51:37.207113 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71446744-3dd9-4599-8849-9b0e9060f417 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:51:37.207387 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71446744-3dd9-4599-8849-9b0e9060f417 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:51:37.218711 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71446744-3dd9-4599-8849-9b0e9060f417 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 10:51:37.224493 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application [None req-71446744-3dd9-4599-8849-9b0e9060f417 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 26 10:51:37.224493 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 10:51:37.224493 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 10:51:37.224493 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 10:51:37.224493 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 10:51:37.224493 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 10:51:37.224493 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 10:51:37.224493 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 10:51:37.224493 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 10:51:37.224493 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 10:51:37.224493 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 10:51:37.224493 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 10:51:37.224493 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 10:51:37.224493 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 10:51:37.224493 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 10:51:37.224493 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 10:51:37.225380 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 10:51:37.225380 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 10:51:37.225380 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 10:51:37.225380 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 10:51:37.225380 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 10:51:37.225380 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 10:51:37.225380 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 10:51:37.225380 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 10:51:37.225380 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 10:51:37.225380 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 10:51:37.225380 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 10:51:37.225380 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application with Lock( Aug 26 10:51:37.225380 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 10:51:37.225380 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._enter() Aug 26 10:51:37.225380 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 10:51:37.225380 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 10:51:37.226583 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 10:51:37.226583 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.creator() Aug 26 10:51:37.226583 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 10:51:37.226583 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application created_value = creator( Aug 26 10:51:37.226583 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 10:51:37.226583 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 10:51:37.226583 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 10:51:37.226583 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 10:51:37.226583 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 10:51:37.226583 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 10:51:37.226583 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 26 10:51:37.226583 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Aug 26 10:51:37.226583 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 143/301] 10.222.0.40 () {56 vars in 1099 bytes} [Fri Aug 26 10:51:37 2022] GET /identity/v3/roles/member => generated 84 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 10:51:37.231709 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321ed80c-6ee0-49f2-8b05-439f5811b2e0 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:51:37.302411 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71446744-3dd9-4599-8849-9b0e9060f417 None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:51:37.338206 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321ed80c-6ee0-49f2-8b05-439f5811b2e0 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:51:37.374403 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c62116e-90c7-44e7-883e-24bf2ed8f87f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:51:37.375046 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c62116e-90c7-44e7-883e-24bf2ed8f87f None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:51:37.375304 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c62116e-90c7-44e7-883e-24bf2ed8f87f None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:51:37.375577 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c62116e-90c7-44e7-883e-24bf2ed8f87f None admin] PATH_INFO: `/v3/roles` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:51:37.376266 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c62116e-90c7-44e7-883e-24bf2ed8f87f None admin] RBAC: Adding query filter params (name=member) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 10:51:37.376570 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c62116e-90c7-44e7-883e-24bf2ed8f87f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:51:37.378277 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c62116e-90c7-44e7-883e-24bf2ed8f87f None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:51:37.390186 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 159/302] 10.222.0.40 () {56 vars in 1101 bytes} [Fri Aug 26 10:51:37 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 10:51:37.390186 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c62116e-90c7-44e7-883e-24bf2ed8f87f None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:51:37.406231 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:51:37.425545 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71446744-3dd9-4599-8849-9b0e9060f417 None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:51:37.461335 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2183f535-87e8-481e-b151-a5041af21c1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:51:37.461960 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2183f535-87e8-481e-b151-a5041af21c1a None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:51:37.462743 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2183f535-87e8-481e-b151-a5041af21c1a None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:51:37.463024 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2183f535-87e8-481e-b151-a5041af21c1a None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:51:37.476415 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2183f535-87e8-481e-b151-a5041af21c1a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 10:51:37.481696 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application [None req-2183f535-87e8-481e-b151-a5041af21c1a None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 26 10:51:37.481696 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 10:51:37.481696 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 10:51:37.481696 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 10:51:37.481696 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 10:51:37.481696 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 10:51:37.481696 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 10:51:37.481696 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 10:51:37.481696 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 10:51:37.481696 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 10:51:37.481696 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 10:51:37.481696 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 10:51:37.481696 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 10:51:37.481696 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 10:51:37.481696 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 10:51:37.481696 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 10:51:37.482619 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 10:51:37.482619 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 10:51:37.482619 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 10:51:37.482619 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 10:51:37.482619 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 10:51:37.482619 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 10:51:37.482619 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 10:51:37.482619 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 10:51:37.482619 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 10:51:37.482619 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 10:51:37.482619 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 10:51:37.482619 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application with Lock( Aug 26 10:51:37.482619 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 10:51:37.482619 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._enter() Aug 26 10:51:37.482619 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 10:51:37.482619 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 10:51:37.485174 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 10:51:37.485174 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.creator() Aug 26 10:51:37.485174 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 10:51:37.485174 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application created_value = creator( Aug 26 10:51:37.485174 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 10:51:37.485174 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 10:51:37.485174 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 10:51:37.485174 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 10:51:37.485174 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 10:51:37.485174 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 10:51:37.485174 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 26 10:51:37.485174 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Aug 26 10:51:37.485174 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 144/303] 10.222.0.40 () {56 vars in 1099 bytes} [Fri Aug 26 10:51:37 2022] GET /identity/v3/roles/reader => generated 84 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 26 10:51:37.485174 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2183f535-87e8-481e-b151-a5041af21c1a None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:51:37.500509 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c62116e-90c7-44e7-883e-24bf2ed8f87f None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:51:37.535025 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35d4e68-68f8-4d13-a8e1-eaac91baaf4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:51:37.535650 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35d4e68-68f8-4d13-a8e1-eaac91baaf4c None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:51:37.535944 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35d4e68-68f8-4d13-a8e1-eaac91baaf4c None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:51:37.536187 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35d4e68-68f8-4d13-a8e1-eaac91baaf4c None admin] PATH_INFO: `/v3/domains/default` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:51:37.537604 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e35d4e68-68f8-4d13-a8e1-eaac91baaf4c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:51:37.539210 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e35d4e68-68f8-4d13-a8e1-eaac91baaf4c None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:51:37.540467 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:51:37.545208 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 160/304] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Aug 26 10:51:37 2022] GET /identity/v3/domains/default => generated 199 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 10:51:37.545208 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35d4e68-68f8-4d13-a8e1-eaac91baaf4c None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:51:37.599492 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2183f535-87e8-481e-b151-a5041af21c1a None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:51:37.628043 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:51:37.639246 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4dcc967-787c-4cba-b292-5d40e39afed0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:51:37.639871 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4dcc967-787c-4cba-b292-5d40e39afed0 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:51:37.640145 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4dcc967-787c-4cba-b292-5d40e39afed0 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:51:37.640409 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4dcc967-787c-4cba-b292-5d40e39afed0 None admin] PATH_INFO: `/v3/users/981035b471a54d77800a40267a6a30ef` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:51:37.641620 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4dcc967-787c-4cba-b292-5d40e39afed0 None admin] RBAC: Authorizing `identity:get_user(user_id=981035b471a54d77800a40267a6a30ef)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:51:37.643764 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4dcc967-787c-4cba-b292-5d40e39afed0 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:51:37.645761 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 145/305] 10.222.0.40 () {56 vars in 1177 bytes} [Fri Aug 26 10:51:37 2022] GET /identity/v3/users/981035b471a54d77800a40267a6a30ef => generated 293 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 10:51:37.646724 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4dcc967-787c-4cba-b292-5d40e39afed0 None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:51:37.654980 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35d4e68-68f8-4d13-a8e1-eaac91baaf4c None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:51:37.688257 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023a36b9-b531-4612-8f60-eb31d56113d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:51:37.688790 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023a36b9-b531-4612-8f60-eb31d56113d6 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:51:37.689064 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023a36b9-b531-4612-8f60-eb31d56113d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:51:37.689335 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023a36b9-b531-4612-8f60-eb31d56113d6 None admin] PATH_INFO: `/v3/roles` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:51:37.690847 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-023a36b9-b531-4612-8f60-eb31d56113d6 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 10:51:37.691149 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-023a36b9-b531-4612-8f60-eb31d56113d6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:51:37.692527 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-023a36b9-b531-4612-8f60-eb31d56113d6 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:51:37.701752 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 161/306] 10.222.0.40 () {56 vars in 1101 bytes} [Fri Aug 26 10:51:37 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 10:51:37.702379 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023a36b9-b531-4612-8f60-eb31d56113d6 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:51:37.762026 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4dcc967-787c-4cba-b292-5d40e39afed0 None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:51:37.781117 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:51:37.800651 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0feb3e55-0447-49bb-a187-fa0e55c83985 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:51:37.801262 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0feb3e55-0447-49bb-a187-fa0e55c83985 None admin] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:51:37.801541 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0feb3e55-0447-49bb-a187-fa0e55c83985 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:51:37.802105 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0feb3e55-0447-49bb-a187-fa0e55c83985 None admin] PATH_INFO: `/v3/groups` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:51:37.802455 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0feb3e55-0447-49bb-a187-fa0e55c83985 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:51:37.803885 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0feb3e55-0447-49bb-a187-fa0e55c83985 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:51:37.807843 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023a36b9-b531-4612-8f60-eb31d56113d6 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:51:37.822834 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 146/307] 10.222.0.40 () {60 vars in 1134 bytes} [Fri Aug 26 10:51:37 2022] POST /identity/v3/groups => generated 229 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 10:51:37.835034 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0feb3e55-0447-49bb-a187-fa0e55c83985 None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:51:37.845849 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb4299d-b86e-4e08-8321-d4ecfce05cce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:51:37.846661 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb4299d-b86e-4e08-8321-d4ecfce05cce None admin] REQUEST_METHOD: `PUT` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:51:37.846941 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb4299d-b86e-4e08-8321-d4ecfce05cce None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:51:37.847211 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb4299d-b86e-4e08-8321-d4ecfce05cce None admin] PATH_INFO: `/v3/system/users/981035b471a54d77800a40267a6a30ef/roles/23e174ce743b46eabe618c212691da55` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:51:37.848774 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb4299d-b86e-4e08-8321-d4ecfce05cce None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=981035b471a54d77800a40267a6a30ef, role_id=23e174ce743b46eabe618c212691da55)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:51:37.850228 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb4299d-b86e-4e08-8321-d4ecfce05cce None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:51:37.865316 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 162/308] 10.222.0.40 () {58 vars in 1334 bytes} [Fri Aug 26 10:51:37 2022] PUT /identity/v3/system/users/981035b471a54d77800a40267a6a30ef/roles/23e174ce743b46eabe618c212691da55 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 10:51:37.988430 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0feb3e55-0447-49bb-a187-fa0e55c83985 None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:51:38.027233 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ae7346-8f69-43d2-b0e2-bd277df75ecf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:51:38.027881 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ae7346-8f69-43d2-b0e2-bd277df75ecf None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:51:38.028155 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ae7346-8f69-43d2-b0e2-bd277df75ecf None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:51:38.028422 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ae7346-8f69-43d2-b0e2-bd277df75ecf None admin] PATH_INFO: `/v3/users/394cc35228574b0fac9f364cb5fba7f0` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:51:38.042592 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4ae7346-8f69-43d2-b0e2-bd277df75ecf None admin] RBAC: Authorizing `identity:get_user(user_id=394cc35228574b0fac9f364cb5fba7f0)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:51:38.044061 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4ae7346-8f69-43d2-b0e2-bd277df75ecf None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:51:38.045885 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 147/309] 10.222.0.40 () {56 vars in 1177 bytes} [Fri Aug 26 10:51:37 2022] GET /identity/v3/users/394cc35228574b0fac9f364cb5fba7f0 => generated 293 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 10:51:38.052316 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb4299d-b86e-4e08-8321-d4ecfce05cce None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:51:38.065266 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:51:38.166232 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb4299d-b86e-4e08-8321-d4ecfce05cce None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:51:38.202309 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de02d06-a5f8-47b6-8280-4079d57a088b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:51:38.202917 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de02d06-a5f8-47b6-8280-4079d57a088b None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:51:38.203188 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de02d06-a5f8-47b6-8280-4079d57a088b None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:51:38.203453 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de02d06-a5f8-47b6-8280-4079d57a088b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:51:38.204181 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de02d06-a5f8-47b6-8280-4079d57a088b None admin] RBAC: Adding query filter params (role.id=dae11e55a3f343ca9954bc6a1e748eed, user.id=394cc35228574b0fac9f364cb5fba7f0, scope.system=all) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 10:51:38.204481 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de02d06-a5f8-47b6-8280-4079d57a088b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:51:38.206469 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de02d06-a5f8-47b6-8280-4079d57a088b None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:51:38.212045 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 163/310] 10.222.0.40 () {56 vars in 1308 bytes} [Fri Aug 26 10:51:38 2022] GET /identity/v3/role_assignments?user.id=394cc35228574b0fac9f364cb5fba7f0&scope.system=all&role.id=dae11e55a3f343ca9954bc6a1e748eed => generated 226 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 10:51:38.256397 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec37bda-4a48-4360-bfc0-ee619277616a None None] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:51:38.256701 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec37bda-4a48-4360-bfc0-ee619277616a None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:51:38.256861 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec37bda-4a48-4360-bfc0-ee619277616a None None] PATH_INFO: `/` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:51:38.258326 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 148/311] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 26 10:51:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 10:51:38.264208 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275c7e0e-3bec-4df7-ac70-1975b03b89d1 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:51:38.264517 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275c7e0e-3bec-4df7-ac70-1975b03b89d1 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:51:38.264816 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275c7e0e-3bec-4df7-ac70-1975b03b89d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:51:38.285443 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-275c7e0e-3bec-4df7-ac70-1975b03b89d1 None None] MFA Rules not processed for user `ca155a48e2024c32af3fd2e19e5fac11`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 10:51:38.286692 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-275c7e0e-3bec-4df7-ac70-1975b03b89d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 10:51:38.367047 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 164/312] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 26 10:51:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 10:51:38.644637 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec37bda-4a48-4360-bfc0-ee619277616a None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:51:38.683031 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:51:38.753769 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec37bda-4a48-4360-bfc0-ee619277616a None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:51:38.788636 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2218b995-e680-4939-8cc0-87d08585f875 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:51:38.789240 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2218b995-e680-4939-8cc0-87d08585f875 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:51:38.789515 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2218b995-e680-4939-8cc0-87d08585f875 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:51:38.789779 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2218b995-e680-4939-8cc0-87d08585f875 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:51:38.801096 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2218b995-e680-4939-8cc0-87d08585f875 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 10:51:38.805963 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application [None req-2218b995-e680-4939-8cc0-87d08585f875 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 26 10:51:38.805963 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 10:51:38.805963 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 10:51:38.805963 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 10:51:38.805963 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 10:51:38.805963 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 10:51:38.805963 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 10:51:38.805963 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 10:51:38.805963 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 10:51:38.805963 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 10:51:38.805963 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 10:51:38.805963 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 10:51:38.805963 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 10:51:38.805963 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 10:51:38.805963 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 10:51:38.805963 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 10:51:38.806853 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 10:51:38.806853 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 10:51:38.806853 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 10:51:38.806853 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 10:51:38.806853 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 10:51:38.806853 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 10:51:38.806853 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 10:51:38.806853 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 10:51:38.806853 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 10:51:38.806853 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 10:51:38.806853 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 10:51:38.806853 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application with Lock( Aug 26 10:51:38.806853 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 10:51:38.806853 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._enter() Aug 26 10:51:38.806853 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 10:51:38.806853 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 10:51:38.807622 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 10:51:38.807622 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.creator() Aug 26 10:51:38.807622 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 10:51:38.807622 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application created_value = creator( Aug 26 10:51:38.807622 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 10:51:38.807622 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 10:51:38.807622 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 10:51:38.807622 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 10:51:38.807622 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 10:51:38.807622 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 10:51:38.807622 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 26 10:51:38.807622 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Aug 26 10:51:38.807622 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 149/313] 10.222.0.40 () {56 vars in 1099 bytes} [Fri Aug 26 10:51:38 2022] GET /identity/v3/roles/reader => generated 84 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 10:51:38.813098 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275c7e0e-3bec-4df7-ac70-1975b03b89d1 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:51:38.815977 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:51:38.926119 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275c7e0e-3bec-4df7-ac70-1975b03b89d1 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:51:38.963102 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f276dae5-8e00-468b-b87e-9a196b46d047 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:51:38.964035 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f276dae5-8e00-468b-b87e-9a196b46d047 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:51:38.964486 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f276dae5-8e00-468b-b87e-9a196b46d047 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:51:38.964926 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f276dae5-8e00-468b-b87e-9a196b46d047 None admin] PATH_INFO: `/v3/roles` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:51:38.965754 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f276dae5-8e00-468b-b87e-9a196b46d047 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 10:51:38.966265 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f276dae5-8e00-468b-b87e-9a196b46d047 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:51:38.967849 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f276dae5-8e00-468b-b87e-9a196b46d047 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:51:38.977774 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 165/314] 10.222.0.40 () {56 vars in 1101 bytes} [Fri Aug 26 10:51:38 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 10:51:38.987590 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2218b995-e680-4939-8cc0-87d08585f875 None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:51:39.098720 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:51:39.101619 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2218b995-e680-4939-8cc0-87d08585f875 None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:51:39.136419 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bdf10d4-e8ad-4308-96d8-b67593f4e163 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:51:39.137045 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bdf10d4-e8ad-4308-96d8-b67593f4e163 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:51:39.137321 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bdf10d4-e8ad-4308-96d8-b67593f4e163 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:51:39.137586 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bdf10d4-e8ad-4308-96d8-b67593f4e163 None admin] PATH_INFO: `/v3/users/455b47e8a24346cc8bb4b1ac0dcb75e3` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:51:39.138797 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bdf10d4-e8ad-4308-96d8-b67593f4e163 None admin] RBAC: Authorizing `identity:get_user(user_id=455b47e8a24346cc8bb4b1ac0dcb75e3)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:51:39.140224 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bdf10d4-e8ad-4308-96d8-b67593f4e163 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:51:39.145997 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 150/315] 10.222.0.40 () {56 vars in 1177 bytes} [Fri Aug 26 10:51:38 2022] GET /identity/v3/users/455b47e8a24346cc8bb4b1ac0dcb75e3 => generated 297 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 10:51:39.147838 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f276dae5-8e00-468b-b87e-9a196b46d047 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:51:39.218535 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:51:39.253035 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f276dae5-8e00-468b-b87e-9a196b46d047 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:51:39.287444 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4f927c-0db8-4b7e-83f5-884e3e9fa734 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:51:39.288280 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4f927c-0db8-4b7e-83f5-884e3e9fa734 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:51:39.288696 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4f927c-0db8-4b7e-83f5-884e3e9fa734 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:51:39.289104 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4f927c-0db8-4b7e-83f5-884e3e9fa734 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:51:39.361100 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f4f927c-0db8-4b7e-83f5-884e3e9fa734 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:51:39.363063 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f4f927c-0db8-4b7e-83f5-884e3e9fa734 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:51:39.425534 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:51:39.464858 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 166/316] 10.222.0.40 () {58 vars in 1282 bytes} [Fri Aug 26 10:51:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 26 10:51:39.470649 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bdf10d4-e8ad-4308-96d8-b67593f4e163 None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:51:39.577466 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bdf10d4-e8ad-4308-96d8-b67593f4e163 None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:51:39.617399 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc45e843-fd93-4d9e-beea-198717765e07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:51:39.618067 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc45e843-fd93-4d9e-beea-198717765e07 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:51:39.618349 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc45e843-fd93-4d9e-beea-198717765e07 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:51:39.618624 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc45e843-fd93-4d9e-beea-198717765e07 None admin] PATH_INFO: `/v3/projects/15667f91e7f6473cb9aab41830d80f5e` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:51:39.619772 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc45e843-fd93-4d9e-beea-198717765e07 None admin] RBAC: Authorizing `identity:get_project(project_id=15667f91e7f6473cb9aab41830d80f5e)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:51:39.621200 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc45e843-fd93-4d9e-beea-198717765e07 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:51:39.623203 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 151/317] 10.222.0.40 () {56 vars in 1186 bytes} [Fri Aug 26 10:51:39 2022] GET /identity/v3/projects/15667f91e7f6473cb9aab41830d80f5e => generated 302 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 10:51:39.629630 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4f927c-0db8-4b7e-83f5-884e3e9fa734 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:51:39.730266 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4f927c-0db8-4b7e-83f5-884e3e9fa734 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:51:39.766460 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a76aaca-40f2-4dee-b893-4083db146252 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:51:39.767450 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a76aaca-40f2-4dee-b893-4083db146252 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:51:39.768005 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a76aaca-40f2-4dee-b893-4083db146252 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:51:39.768470 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a76aaca-40f2-4dee-b893-4083db146252 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:51:39.769384 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a76aaca-40f2-4dee-b893-4083db146252 None admin] RBAC: Adding query filter params (role.id=dae11e55a3f343ca9954bc6a1e748eed, scope.project.id=15667f91e7f6473cb9aab41830d80f5e, user.id=455b47e8a24346cc8bb4b1ac0dcb75e3) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 10:51:39.769882 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a76aaca-40f2-4dee-b893-4083db146252 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:51:39.771538 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a76aaca-40f2-4dee-b893-4083db146252 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:51:39.778352 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 167/318] 10.222.0.40 () {56 vars in 1374 bytes} [Fri Aug 26 10:51:39 2022] GET /identity/v3/role_assignments?user.id=455b47e8a24346cc8bb4b1ac0dcb75e3&scope.project.id=15667f91e7f6473cb9aab41830d80f5e&role.id=dae11e55a3f343ca9954bc6a1e748eed => generated 608 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 10:51:39.881867 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96670cfe-3ddc-4e97-844b-414820da9fe7 None None] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:51:39.882393 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96670cfe-3ddc-4e97-844b-414820da9fe7 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:51:39.882469 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96670cfe-3ddc-4e97-844b-414820da9fe7 None None] PATH_INFO: `/` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:51:39.884100 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 152/319] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 26 10:51:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 10:51:39.889876 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5653ab43-76c4-4840-8fb6-392185b4a58e None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:51:39.890515 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5653ab43-76c4-4840-8fb6-392185b4a58e None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:51:39.890998 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5653ab43-76c4-4840-8fb6-392185b4a58e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:51:39.910862 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-5653ab43-76c4-4840-8fb6-392185b4a58e None None] MFA Rules not processed for user `ca155a48e2024c32af3fd2e19e5fac11`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 10:51:39.912471 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-5653ab43-76c4-4840-8fb6-392185b4a58e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 10:51:39.939497 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:51:39.946153 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1704238-5971-4332-9d56-81200b2cd2a9 None None] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:51:39.946803 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1704238-5971-4332-9d56-81200b2cd2a9 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:51:39.947301 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1704238-5971-4332-9d56-81200b2cd2a9 None None] PATH_INFO: `/` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:51:39.948651 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 153/320] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 26 10:51:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 10:51:39.954611 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6564d5d-3787-4d77-adc6-8205814881a3 None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:51:39.954888 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6564d5d-3787-4d77-adc6-8205814881a3 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:51:39.955291 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6564d5d-3787-4d77-adc6-8205814881a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:51:39.978408 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-d6564d5d-3787-4d77-adc6-8205814881a3 None None] MFA Rules not processed for user `ca155a48e2024c32af3fd2e19e5fac11`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 10:51:39.979823 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-d6564d5d-3787-4d77-adc6-8205814881a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 10:51:39.992481 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 168/321] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 26 10:51:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 10:51:40.013908 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:51:40.065488 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 154/322] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 26 10:51:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 26 10:51:40.218846 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f8837d-ce55-48f3-8d7e-a174bab35500 None None] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:51:40.219353 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f8837d-ce55-48f3-8d7e-a174bab35500 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:51:40.219451 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f8837d-ce55-48f3-8d7e-a174bab35500 None None] PATH_INFO: `/` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:51:40.221480 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 169/323] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 26 10:51:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 10:51:40.226494 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f6cd97-e3f2-4f24-8e3d-d6ba7c16baf9 None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:51:40.226852 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f6cd97-e3f2-4f24-8e3d-d6ba7c16baf9 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:51:40.227197 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f6cd97-e3f2-4f24-8e3d-d6ba7c16baf9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:51:40.245403 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-93f6cd97-e3f2-4f24-8e3d-d6ba7c16baf9 None None] MFA Rules not processed for user `ca155a48e2024c32af3fd2e19e5fac11`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 10:51:40.246539 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-93f6cd97-e3f2-4f24-8e3d-d6ba7c16baf9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 10:51:40.259475 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f8837d-ce55-48f3-8d7e-a174bab35500 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:51:40.321824 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 155/324] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 26 10:51:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 10:51:40.339026 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f6cd97-e3f2-4f24-8e3d-d6ba7c16baf9 None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:51:40.368302 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f8837d-ce55-48f3-8d7e-a174bab35500 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:51:40.403973 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d647028d-7142-46df-8ab9-0a10bdd4f526 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:51:40.404594 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d647028d-7142-46df-8ab9-0a10bdd4f526 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:51:40.404868 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d647028d-7142-46df-8ab9-0a10bdd4f526 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:51:40.405135 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d647028d-7142-46df-8ab9-0a10bdd4f526 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:51:40.416756 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d647028d-7142-46df-8ab9-0a10bdd4f526 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 10:51:40.421837 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application [None req-d647028d-7142-46df-8ab9-0a10bdd4f526 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 26 10:51:40.421837 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 10:51:40.421837 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 10:51:40.421837 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 10:51:40.421837 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 10:51:40.421837 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 10:51:40.421837 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 10:51:40.421837 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 10:51:40.421837 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 10:51:40.421837 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 10:51:40.421837 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 10:51:40.421837 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 10:51:40.421837 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 10:51:40.421837 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 10:51:40.421837 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 10:51:40.421837 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 10:51:40.422734 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 10:51:40.422734 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 10:51:40.422734 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 10:51:40.422734 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 10:51:40.422734 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 10:51:40.422734 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 10:51:40.422734 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 10:51:40.422734 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 10:51:40.422734 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 10:51:40.422734 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 10:51:40.422734 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 10:51:40.422734 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application with Lock( Aug 26 10:51:40.422734 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 10:51:40.422734 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._enter() Aug 26 10:51:40.422734 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 10:51:40.422734 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 10:51:40.423574 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 10:51:40.423574 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.creator() Aug 26 10:51:40.423574 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 10:51:40.423574 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application created_value = creator( Aug 26 10:51:40.423574 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 10:51:40.423574 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 10:51:40.423574 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 10:51:40.423574 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 10:51:40.423574 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 10:51:40.423574 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 10:51:40.423574 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 26 10:51:40.423574 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Aug 26 10:51:40.423574 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 170/325] 10.222.0.40 () {56 vars in 1099 bytes} [Fri Aug 26 10:51:40 2022] GET /identity/v3/roles/member => generated 84 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 26 10:51:40.429085 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d647028d-7142-46df-8ab9-0a10bdd4f526 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:51:40.434807 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:51:40.444791 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f6cd97-e3f2-4f24-8e3d-d6ba7c16baf9 None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:51:40.450013 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:51:40.480369 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e88632b-58b0-471f-b160-4ef89db921dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:51:40.481004 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e88632b-58b0-471f-b160-4ef89db921dc None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:51:40.481287 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e88632b-58b0-471f-b160-4ef89db921dc None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:51:40.481557 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e88632b-58b0-471f-b160-4ef89db921dc None admin] PATH_INFO: `/v3/domains/default` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:51:40.482689 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e88632b-58b0-471f-b160-4ef89db921dc None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:51:40.484390 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e88632b-58b0-471f-b160-4ef89db921dc None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:51:40.486006 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 156/326] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Aug 26 10:51:40 2022] GET /identity/v3/domains/default => generated 199 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 10:51:40.491497 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e88632b-58b0-471f-b160-4ef89db921dc None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:51:40.543328 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d647028d-7142-46df-8ab9-0a10bdd4f526 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:51:40.582605 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ead15c7-6bc2-4439-a452-9bc43f9d5d10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:51:40.583499 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ead15c7-6bc2-4439-a452-9bc43f9d5d10 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:51:40.583777 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ead15c7-6bc2-4439-a452-9bc43f9d5d10 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:51:40.584045 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ead15c7-6bc2-4439-a452-9bc43f9d5d10 None admin] PATH_INFO: `/v3/roles` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:51:40.584705 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ead15c7-6bc2-4439-a452-9bc43f9d5d10 None admin] RBAC: Adding query filter params (name=member) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 10:51:40.585000 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ead15c7-6bc2-4439-a452-9bc43f9d5d10 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:51:40.586882 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ead15c7-6bc2-4439-a452-9bc43f9d5d10 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:51:40.595233 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 171/327] 10.222.0.40 () {56 vars in 1101 bytes} [Fri Aug 26 10:51:40 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 10:51:40.596655 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ead15c7-6bc2-4439-a452-9bc43f9d5d10 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:51:40.598914 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e88632b-58b0-471f-b160-4ef89db921dc None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:51:40.632768 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af1a6d3-5af0-4d7c-98fd-4b48a641830a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:51:40.633362 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af1a6d3-5af0-4d7c-98fd-4b48a641830a None admin] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:51:40.633634 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af1a6d3-5af0-4d7c-98fd-4b48a641830a None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:51:40.633902 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af1a6d3-5af0-4d7c-98fd-4b48a641830a None admin] PATH_INFO: `/v3/groups` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:51:40.634547 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af1a6d3-5af0-4d7c-98fd-4b48a641830a None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:51:40.635968 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af1a6d3-5af0-4d7c-98fd-4b48a641830a None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:51:40.651856 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 157/328] 10.222.0.40 () {60 vars in 1134 bytes} [Fri Aug 26 10:51:40 2022] POST /identity/v3/groups => generated 226 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 26 10:51:40.659351 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af1a6d3-5af0-4d7c-98fd-4b48a641830a None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:51:40.685428 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:51:40.690013 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:51:40.712909 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ead15c7-6bc2-4439-a452-9bc43f9d5d10 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:51:40.749357 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f273c82b-268f-4ead-96e0-6e4d718a823f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:51:40.750205 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f273c82b-268f-4ead-96e0-6e4d718a823f None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:51:40.750615 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f273c82b-268f-4ead-96e0-6e4d718a823f None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:51:40.750989 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f273c82b-268f-4ead-96e0-6e4d718a823f None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:51:40.763276 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f273c82b-268f-4ead-96e0-6e4d718a823f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 10:51:40.768985 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application [None req-f273c82b-268f-4ead-96e0-6e4d718a823f None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 26 10:51:40.768985 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 10:51:40.768985 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 10:51:40.768985 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 10:51:40.768985 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 10:51:40.768985 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 10:51:40.768985 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 10:51:40.768985 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 10:51:40.768985 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 10:51:40.768985 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 10:51:40.768985 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 10:51:40.768985 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 10:51:40.768985 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 10:51:40.768985 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 10:51:40.768985 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 10:51:40.768985 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 10:51:40.770134 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 10:51:40.770134 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 10:51:40.770134 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 10:51:40.770134 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 10:51:40.770134 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 10:51:40.770134 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 10:51:40.770134 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 10:51:40.770134 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 10:51:40.770134 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 10:51:40.770134 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 10:51:40.770134 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 10:51:40.770134 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application with Lock( Aug 26 10:51:40.770134 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 10:51:40.770134 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._enter() Aug 26 10:51:40.770134 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 10:51:40.770134 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 10:51:40.771135 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 10:51:40.771135 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.creator() Aug 26 10:51:40.771135 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 10:51:40.771135 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application created_value = creator( Aug 26 10:51:40.771135 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 10:51:40.771135 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 10:51:40.771135 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 10:51:40.771135 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 10:51:40.771135 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 10:51:40.771135 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 10:51:40.771135 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 26 10:51:40.771135 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Aug 26 10:51:40.771135 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 172/329] 10.222.0.40 () {56 vars in 1099 bytes} [Fri Aug 26 10:51:40 2022] GET /identity/v3/roles/reader => generated 84 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 10:51:40.778058 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f273c82b-268f-4ead-96e0-6e4d718a823f None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:51:40.794881 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af1a6d3-5af0-4d7c-98fd-4b48a641830a None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:51:40.833619 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-290ef14d-0bbb-4f33-9829-9f2e27cb8c4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:51:40.834527 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290ef14d-0bbb-4f33-9829-9f2e27cb8c4b None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:51:40.834961 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290ef14d-0bbb-4f33-9829-9f2e27cb8c4b None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:51:40.835231 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290ef14d-0bbb-4f33-9829-9f2e27cb8c4b None admin] PATH_INFO: `/v3/users/981035b471a54d77800a40267a6a30ef` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:51:40.836384 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-290ef14d-0bbb-4f33-9829-9f2e27cb8c4b None admin] RBAC: Authorizing `identity:get_user(user_id=981035b471a54d77800a40267a6a30ef)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:51:40.837853 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-290ef14d-0bbb-4f33-9829-9f2e27cb8c4b None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:51:40.840197 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 158/330] 10.222.0.40 () {56 vars in 1177 bytes} [Fri Aug 26 10:51:40 2022] GET /identity/v3/users/981035b471a54d77800a40267a6a30ef => generated 293 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 10:51:40.845304 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-290ef14d-0bbb-4f33-9829-9f2e27cb8c4b None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:51:40.916340 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f273c82b-268f-4ead-96e0-6e4d718a823f None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:51:40.966754 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2826cebf-3231-43f8-8b4e-3f5dc1c65578 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:51:40.967562 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2826cebf-3231-43f8-8b4e-3f5dc1c65578 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:51:40.967912 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2826cebf-3231-43f8-8b4e-3f5dc1c65578 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:51:40.968007 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:51:40.968398 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2826cebf-3231-43f8-8b4e-3f5dc1c65578 None admin] PATH_INFO: `/v3/roles` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:51:40.969250 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2826cebf-3231-43f8-8b4e-3f5dc1c65578 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 10:51:40.969761 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2826cebf-3231-43f8-8b4e-3f5dc1c65578 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:51:40.972222 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2826cebf-3231-43f8-8b4e-3f5dc1c65578 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:51:40.980954 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 173/331] 10.222.0.40 () {56 vars in 1101 bytes} [Fri Aug 26 10:51:40 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 10:51:40.984290 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-290ef14d-0bbb-4f33-9829-9f2e27cb8c4b None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:51:40.987838 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2826cebf-3231-43f8-8b4e-3f5dc1c65578 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:51:40.992036 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:51:41.038475 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48de9728-24dd-4543-aebc-9a83f37bd001 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:51:41.038475 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48de9728-24dd-4543-aebc-9a83f37bd001 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:51:41.038778 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48de9728-24dd-4543-aebc-9a83f37bd001 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:51:41.039001 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48de9728-24dd-4543-aebc-9a83f37bd001 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:51:41.039803 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48de9728-24dd-4543-aebc-9a83f37bd001 None admin] RBAC: Adding query filter params (role.id=23e174ce743b46eabe618c212691da55, user.id=981035b471a54d77800a40267a6a30ef, scope.system=all) {{(pid=81423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 10:51:41.040110 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48de9728-24dd-4543-aebc-9a83f37bd001 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:51:41.041868 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48de9728-24dd-4543-aebc-9a83f37bd001 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:51:41.056921 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 159/332] 10.222.0.40 () {56 vars in 1308 bytes} [Fri Aug 26 10:51:40 2022] GET /identity/v3/role_assignments?user.id=981035b471a54d77800a40267a6a30ef&scope.system=all&role.id=23e174ce743b46eabe618c212691da55 => generated 510 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 10:51:41.120482 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2826cebf-3231-43f8-8b4e-3f5dc1c65578 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:51:41.167281 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97bead0-228f-4862-80a1-97ab8047f9d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:51:41.167958 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97bead0-228f-4862-80a1-97ab8047f9d5 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:51:41.168226 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97bead0-228f-4862-80a1-97ab8047f9d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:51:41.168500 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97bead0-228f-4862-80a1-97ab8047f9d5 None admin] PATH_INFO: `/v3/users/394cc35228574b0fac9f364cb5fba7f0` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:51:41.169693 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a97bead0-228f-4862-80a1-97ab8047f9d5 None admin] RBAC: Authorizing `identity:get_user(user_id=394cc35228574b0fac9f364cb5fba7f0)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:51:41.171377 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a97bead0-228f-4862-80a1-97ab8047f9d5 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:51:41.173540 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 174/333] 10.222.0.40 () {56 vars in 1177 bytes} [Fri Aug 26 10:51:40 2022] GET /identity/v3/users/394cc35228574b0fac9f364cb5fba7f0 => generated 293 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 10:51:41.179079 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48de9728-24dd-4543-aebc-9a83f37bd001 None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:51:41.300659 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48de9728-24dd-4543-aebc-9a83f37bd001 None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:51:41.342241 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d247c597-4788-4f04-ab38-20d24c98a7e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:51:41.342910 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d247c597-4788-4f04-ab38-20d24c98a7e1 None admin] REQUEST_METHOD: `PUT` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:51:41.343192 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d247c597-4788-4f04-ab38-20d24c98a7e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:51:41.343467 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d247c597-4788-4f04-ab38-20d24c98a7e1 None admin] PATH_INFO: `/v3/system/users/394cc35228574b0fac9f364cb5fba7f0/roles/dae11e55a3f343ca9954bc6a1e748eed` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:51:41.344973 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d247c597-4788-4f04-ab38-20d24c98a7e1 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=394cc35228574b0fac9f364cb5fba7f0, role_id=dae11e55a3f343ca9954bc6a1e748eed)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:51:41.346682 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d247c597-4788-4f04-ab38-20d24c98a7e1 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:51:41.356117 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 160/334] 10.222.0.40 () {58 vars in 1334 bytes} [Fri Aug 26 10:51:41 2022] PUT /identity/v3/system/users/394cc35228574b0fac9f364cb5fba7f0/roles/dae11e55a3f343ca9954bc6a1e748eed => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 10:51:42.915907 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b3004c-ad04-486b-83f0-afc4161b3f8d None None] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:51:42.916434 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b3004c-ad04-486b-83f0-afc4161b3f8d None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:51:42.916434 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b3004c-ad04-486b-83f0-afc4161b3f8d None None] PATH_INFO: `/` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:51:42.918067 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 175/335] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 26 10:51:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 10:51:42.923657 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbde4561-76ea-4568-bea9-4f78d6e801c0 None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:51:42.924125 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbde4561-76ea-4568-bea9-4f78d6e801c0 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:51:42.924222 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbde4561-76ea-4568-bea9-4f78d6e801c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:51:42.951222 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-fbde4561-76ea-4568-bea9-4f78d6e801c0 None None] MFA Rules not processed for user `ca155a48e2024c32af3fd2e19e5fac11`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 10:51:42.952426 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-fbde4561-76ea-4568-bea9-4f78d6e801c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 10:51:42.959185 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:51:43.038664 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 161/336] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 26 10:51:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 10:51:43.590870 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b3004c-ad04-486b-83f0-afc4161b3f8d None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:51:43.645807 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5937f2-7be3-4671-8c70-a907741083f9 None None] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:51:43.646657 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5937f2-7be3-4671-8c70-a907741083f9 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:51:43.646937 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5937f2-7be3-4671-8c70-a907741083f9 None None] PATH_INFO: `/` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:51:43.653176 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 162/337] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 26 10:51:43 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 10:51:43.654770 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c2ba1f-a036-4b4b-91c9-f76ca88bc4ed None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:51:43.655049 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c2ba1f-a036-4b4b-91c9-f76ca88bc4ed None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:51:43.655838 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c2ba1f-a036-4b4b-91c9-f76ca88bc4ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:51:43.687672 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-b9c2ba1f-a036-4b4b-91c9-f76ca88bc4ed None None] MFA Rules not processed for user `ca155a48e2024c32af3fd2e19e5fac11`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 10:51:43.688883 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-b9c2ba1f-a036-4b4b-91c9-f76ca88bc4ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 10:51:43.698469 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:51:43.771559 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b3004c-ad04-486b-83f0-afc4161b3f8d None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:51:43.796118 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 163/338] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 26 10:51:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 26 10:51:43.839596 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30de507-8418-48d0-978d-bf308f86a1fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:51:43.840244 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30de507-8418-48d0-978d-bf308f86a1fc None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:51:43.840662 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30de507-8418-48d0-978d-bf308f86a1fc None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:51:43.840990 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30de507-8418-48d0-978d-bf308f86a1fc None admin] PATH_INFO: `/v3/roles/member` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:51:43.861768 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f30de507-8418-48d0-978d-bf308f86a1fc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 10:51:43.866887 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application [None req-f30de507-8418-48d0-978d-bf308f86a1fc None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 26 10:51:43.866887 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 10:51:43.866887 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 10:51:43.866887 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 10:51:43.866887 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 10:51:43.866887 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 10:51:43.866887 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 10:51:43.866887 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 10:51:43.866887 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 10:51:43.866887 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 10:51:43.866887 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 10:51:43.866887 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 10:51:43.866887 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 10:51:43.866887 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 10:51:43.866887 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 10:51:43.866887 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 10:51:43.867768 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 10:51:43.867768 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 10:51:43.867768 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 10:51:43.867768 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 10:51:43.867768 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 10:51:43.867768 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 10:51:43.867768 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 10:51:43.867768 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 10:51:43.867768 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 10:51:43.867768 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 10:51:43.867768 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 10:51:43.867768 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application with Lock( Aug 26 10:51:43.867768 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 10:51:43.867768 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._enter() Aug 26 10:51:43.867768 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 10:51:43.867768 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 10:51:43.868563 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 10:51:43.868563 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.creator() Aug 26 10:51:43.868563 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 10:51:43.868563 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application created_value = creator( Aug 26 10:51:43.868563 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 10:51:43.868563 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 10:51:43.868563 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 10:51:43.868563 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 10:51:43.868563 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 10:51:43.868563 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 10:51:43.868563 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 26 10:51:43.868563 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Aug 26 10:51:43.868563 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 176/339] 10.222.0.40 () {56 vars in 1099 bytes} [Fri Aug 26 10:51:43 2022] GET /identity/v3/roles/member => generated 84 bytes in 278 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 10:51:43.875783 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c2ba1f-a036-4b4b-91c9-f76ca88bc4ed None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:51:43.975283 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b1e17a-f59b-42f4-8266-ab4e2ae40b20 None None] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:51:43.975803 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b1e17a-f59b-42f4-8266-ab4e2ae40b20 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:51:43.975885 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b1e17a-f59b-42f4-8266-ab4e2ae40b20 None None] PATH_INFO: `/` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:51:43.976925 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:51:43.977573 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 177/340] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 26 10:51:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 10:51:43.994336 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a577b19-055b-4fcb-afe2-dbeb698d6237 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:51:43.994971 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a577b19-055b-4fcb-afe2-dbeb698d6237 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:51:43.995469 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a577b19-055b-4fcb-afe2-dbeb698d6237 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:51:44.024661 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c2ba1f-a036-4b4b-91c9-f76ca88bc4ed None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:51:44.030965 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-4a577b19-055b-4fcb-afe2-dbeb698d6237 None None] MFA Rules not processed for user `ca155a48e2024c32af3fd2e19e5fac11`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 10:51:44.032451 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-4a577b19-055b-4fcb-afe2-dbeb698d6237 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 10:51:44.073097 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c775d049-4753-47fb-ab88-67618080a100 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:51:44.073860 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c775d049-4753-47fb-ab88-67618080a100 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:51:44.074170 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c775d049-4753-47fb-ab88-67618080a100 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:51:44.074438 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c775d049-4753-47fb-ab88-67618080a100 None admin] PATH_INFO: `/v3/roles` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:51:44.075193 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c775d049-4753-47fb-ab88-67618080a100 None admin] RBAC: Adding query filter params (name=member) {{(pid=81423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 10:51:44.075491 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c775d049-4753-47fb-ab88-67618080a100 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:51:44.077126 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c775d049-4753-47fb-ab88-67618080a100 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:51:44.093888 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 164/341] 10.222.0.40 () {56 vars in 1101 bytes} [Fri Aug 26 10:51:43 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 10:51:44.100522 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c775d049-4753-47fb-ab88-67618080a100 None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:51:44.156331 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 178/342] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 26 10:51:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 26 10:51:44.249506 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d54e384-c1ee-4c26-a156-b0d1f0d37d9e None None] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:51:44.249762 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c775d049-4753-47fb-ab88-67618080a100 None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:51:44.249906 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d54e384-c1ee-4c26-a156-b0d1f0d37d9e None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:51:44.249906 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d54e384-c1ee-4c26-a156-b0d1f0d37d9e None None] PATH_INFO: `/` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:51:44.250488 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 179/343] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 26 10:51:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 10:51:44.259519 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b4e708-7349-40f5-bfc2-95830bec4cd4 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:51:44.260121 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b4e708-7349-40f5-bfc2-95830bec4cd4 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:51:44.260601 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b4e708-7349-40f5-bfc2-95830bec4cd4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:51:44.292875 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:51:44.300648 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-d3b4e708-7349-40f5-bfc2-95830bec4cd4 None None] MFA Rules not processed for user `ca155a48e2024c32af3fd2e19e5fac11`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 10:51:44.301044 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c50288b-8297-4313-beca-d9ef6e12a8ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:51:44.301708 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c50288b-8297-4313-beca-d9ef6e12a8ae None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:51:44.302000 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c50288b-8297-4313-beca-d9ef6e12a8ae None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:51:44.302279 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c50288b-8297-4313-beca-d9ef6e12a8ae None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:51:44.303057 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-d3b4e708-7349-40f5-bfc2-95830bec4cd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 10:51:44.423244 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:51:44.435208 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c50288b-8297-4313-beca-d9ef6e12a8ae None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:51:44.437908 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c50288b-8297-4313-beca-d9ef6e12a8ae None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:51:44.459468 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 180/344] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 26 10:51:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) Aug 26 10:51:44.460320 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b4e708-7349-40f5-bfc2-95830bec4cd4 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:51:44.619281 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 165/345] 10.222.0.40 () {58 vars in 1282 bytes} [Fri Aug 26 10:51:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 520 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 26 10:51:44.625021 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c50288b-8297-4313-beca-d9ef6e12a8ae None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:51:44.634138 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b4e708-7349-40f5-bfc2-95830bec4cd4 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:51:44.674811 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:51:44.679665 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f26b1a7-671a-4567-ae65-8b9e69db8f9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:51:44.680322 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f26b1a7-671a-4567-ae65-8b9e69db8f9b None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:51:44.680604 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f26b1a7-671a-4567-ae65-8b9e69db8f9b None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:51:44.680881 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f26b1a7-671a-4567-ae65-8b9e69db8f9b None admin] PATH_INFO: `/v3/roles/member` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:51:44.706556 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f26b1a7-671a-4567-ae65-8b9e69db8f9b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 10:51:44.711721 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application [None req-1f26b1a7-671a-4567-ae65-8b9e69db8f9b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 26 10:51:44.711721 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 10:51:44.711721 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 10:51:44.711721 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 10:51:44.711721 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 10:51:44.711721 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 10:51:44.711721 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 10:51:44.711721 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 10:51:44.711721 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 10:51:44.711721 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 10:51:44.711721 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 10:51:44.711721 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 10:51:44.711721 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 10:51:44.711721 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 10:51:44.711721 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 10:51:44.711721 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 10:51:44.712804 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 10:51:44.712804 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 10:51:44.712804 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 10:51:44.712804 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 10:51:44.712804 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 10:51:44.712804 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 10:51:44.712804 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 10:51:44.712804 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 10:51:44.712804 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 10:51:44.712804 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 10:51:44.712804 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 10:51:44.712804 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application with Lock( Aug 26 10:51:44.712804 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 10:51:44.712804 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._enter() Aug 26 10:51:44.712804 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 10:51:44.712804 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 10:51:44.713781 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 10:51:44.713781 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.creator() Aug 26 10:51:44.713781 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 10:51:44.713781 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application created_value = creator( Aug 26 10:51:44.713781 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 10:51:44.713781 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 10:51:44.713781 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 10:51:44.713781 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 10:51:44.713781 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 10:51:44.713781 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 10:51:44.713781 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 26 10:51:44.713781 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Aug 26 10:51:44.716060 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 181/346] 10.222.0.40 () {56 vars in 1099 bytes} [Fri Aug 26 10:51:44 2022] GET /identity/v3/roles/member => generated 84 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 10:51:44.717075 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f26b1a7-671a-4567-ae65-8b9e69db8f9b None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:51:44.750547 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c50288b-8297-4313-beca-d9ef6e12a8ae None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:51:44.790644 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600177c9-2873-48f9-93f4-5f6a3edde710 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:51:44.791657 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600177c9-2873-48f9-93f4-5f6a3edde710 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:51:44.792131 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600177c9-2873-48f9-93f4-5f6a3edde710 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:51:44.792592 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600177c9-2873-48f9-93f4-5f6a3edde710 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:51:44.815214 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-600177c9-2873-48f9-93f4-5f6a3edde710 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 10:51:44.817430 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:51:44.822711 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application [None req-600177c9-2873-48f9-93f4-5f6a3edde710 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 26 10:51:44.822711 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 10:51:44.822711 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 10:51:44.822711 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 10:51:44.822711 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 10:51:44.822711 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 10:51:44.822711 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 10:51:44.822711 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 10:51:44.822711 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 10:51:44.822711 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 10:51:44.822711 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 10:51:44.822711 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 10:51:44.822711 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 10:51:44.822711 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 10:51:44.822711 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 10:51:44.822711 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 10:51:44.823877 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 10:51:44.823877 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 10:51:44.823877 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 10:51:44.823877 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 10:51:44.823877 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 10:51:44.823877 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 10:51:44.823877 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 10:51:44.823877 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 10:51:44.823877 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 10:51:44.823877 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 10:51:44.823877 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 10:51:44.823877 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application with Lock( Aug 26 10:51:44.823877 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 10:51:44.823877 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._enter() Aug 26 10:51:44.823877 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 10:51:44.823877 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 10:51:44.824746 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 10:51:44.824746 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.creator() Aug 26 10:51:44.824746 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 10:51:44.824746 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application created_value = creator( Aug 26 10:51:44.824746 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 10:51:44.824746 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 10:51:44.824746 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 10:51:44.824746 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 10:51:44.824746 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 10:51:44.824746 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 10:51:44.824746 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 26 10:51:44.824746 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Aug 26 10:51:44.827854 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 166/347] 10.222.0.40 () {56 vars in 1114 bytes} [Fri Aug 26 10:51:44 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 10:51:44.828880 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600177c9-2873-48f9-93f4-5f6a3edde710 None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:51:44.884862 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f26b1a7-671a-4567-ae65-8b9e69db8f9b None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:51:44.927369 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a735836c-ac5d-4384-8662-98d4cc3e7c58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:51:44.928598 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a735836c-ac5d-4384-8662-98d4cc3e7c58 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:51:44.929709 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a735836c-ac5d-4384-8662-98d4cc3e7c58 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:51:44.930339 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a735836c-ac5d-4384-8662-98d4cc3e7c58 None admin] PATH_INFO: `/v3/projects/da070dea52de4d71952ed50827d44a1c` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:51:44.931971 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a735836c-ac5d-4384-8662-98d4cc3e7c58 None admin] RBAC: Authorizing `identity:get_project(project_id=da070dea52de4d71952ed50827d44a1c)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:51:44.933838 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a735836c-ac5d-4384-8662-98d4cc3e7c58 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:51:44.936622 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 182/348] 10.222.0.40 () {56 vars in 1186 bytes} [Fri Aug 26 10:51:44 2022] GET /identity/v3/projects/da070dea52de4d71952ed50827d44a1c => generated 298 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 10:51:44.938195 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc8f415-075f-4de2-89d2-10f4f85590ab None None] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:51:44.938706 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc8f415-075f-4de2-89d2-10f4f85590ab None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:51:44.939174 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc8f415-075f-4de2-89d2-10f4f85590ab None None] PATH_INFO: `/` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:51:44.945128 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 183/349] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 26 10:51:44 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 10:51:44.945128 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc8f415-075f-4de2-89d2-10f4f85590ab None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:51:44.964231 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:51:44.976231 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600177c9-2873-48f9-93f4-5f6a3edde710 None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:51:45.026778 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c817a0d-155e-485d-b7ed-aa18e682c44b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:51:45.028340 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c817a0d-155e-485d-b7ed-aa18e682c44b None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:51:45.028794 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c817a0d-155e-485d-b7ed-aa18e682c44b None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:51:45.029232 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c817a0d-155e-485d-b7ed-aa18e682c44b None admin] PATH_INFO: `/v3/roles` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:51:45.030107 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c817a0d-155e-485d-b7ed-aa18e682c44b None admin] RBAC: Adding query filter params (name=member) {{(pid=81423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 10:51:45.030575 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c817a0d-155e-485d-b7ed-aa18e682c44b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:51:45.032098 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c817a0d-155e-485d-b7ed-aa18e682c44b None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:51:45.042113 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 167/350] 10.222.0.40 () {56 vars in 1101 bytes} [Fri Aug 26 10:51:44 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 10:51:45.043126 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c817a0d-155e-485d-b7ed-aa18e682c44b None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:51:45.081050 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc8f415-075f-4de2-89d2-10f4f85590ab None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:51:45.106697 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:51:45.130557 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da25955-87fc-443f-87c5-d74960bd3cad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:51:45.131335 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da25955-87fc-443f-87c5-d74960bd3cad None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:51:45.131618 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da25955-87fc-443f-87c5-d74960bd3cad None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:51:45.131889 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da25955-87fc-443f-87c5-d74960bd3cad None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:51:45.144814 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da25955-87fc-443f-87c5-d74960bd3cad None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 10:51:45.149616 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application [None req-3da25955-87fc-443f-87c5-d74960bd3cad None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 26 10:51:45.149616 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 10:51:45.149616 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 10:51:45.149616 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 10:51:45.149616 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 10:51:45.149616 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 10:51:45.149616 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 10:51:45.149616 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 10:51:45.149616 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 10:51:45.149616 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 10:51:45.149616 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 10:51:45.149616 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 10:51:45.149616 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 10:51:45.149616 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 10:51:45.149616 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 10:51:45.149616 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 10:51:45.150597 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 10:51:45.150597 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 10:51:45.150597 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 10:51:45.150597 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 10:51:45.150597 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 10:51:45.150597 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 10:51:45.150597 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 10:51:45.150597 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 10:51:45.150597 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 10:51:45.150597 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 10:51:45.150597 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 10:51:45.150597 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application with Lock( Aug 26 10:51:45.150597 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 10:51:45.150597 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._enter() Aug 26 10:51:45.150597 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 10:51:45.150597 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 10:51:45.151586 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 10:51:45.151586 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.creator() Aug 26 10:51:45.151586 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 10:51:45.151586 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application created_value = creator( Aug 26 10:51:45.151586 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 10:51:45.151586 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 10:51:45.151586 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 10:51:45.151586 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 10:51:45.151586 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 10:51:45.151586 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 10:51:45.151586 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 26 10:51:45.151586 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Aug 26 10:51:45.151586 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 184/351] 10.222.0.40 () {56 vars in 1099 bytes} [Fri Aug 26 10:51:44 2022] GET /identity/v3/roles/reader => generated 84 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 10:51:45.152253 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da25955-87fc-443f-87c5-d74960bd3cad None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:51:45.177811 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c817a0d-155e-485d-b7ed-aa18e682c44b None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:51:45.223503 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2d8fac-f64d-44f2-b988-8ee614313ad5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:51:45.225458 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2d8fac-f64d-44f2-b988-8ee614313ad5 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:51:45.225458 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2d8fac-f64d-44f2-b988-8ee614313ad5 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:51:45.225458 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2d8fac-f64d-44f2-b988-8ee614313ad5 None admin] PATH_INFO: `/v3/roles` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:51:45.225458 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d2d8fac-f64d-44f2-b988-8ee614313ad5 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 10:51:45.225756 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d2d8fac-f64d-44f2-b988-8ee614313ad5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:51:45.226955 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d2d8fac-f64d-44f2-b988-8ee614313ad5 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:51:45.242308 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 168/352] 10.222.0.40 () {56 vars in 1111 bytes} [Fri Aug 26 10:51:45 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 10:51:45.243576 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b84f976-ec9e-4ea6-9ca5-08614cc51226 None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:51:45.243856 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b84f976-ec9e-4ea6-9ca5-08614cc51226 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:51:45.244184 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b84f976-ec9e-4ea6-9ca5-08614cc51226 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:51:45.272796 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-6b84f976-ec9e-4ea6-9ca5-08614cc51226 None None] MFA Rules not processed for user `ca155a48e2024c32af3fd2e19e5fac11`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 10:51:45.274347 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-6b84f976-ec9e-4ea6-9ca5-08614cc51226 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 10:51:45.288187 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da25955-87fc-443f-87c5-d74960bd3cad None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:51:45.290656 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:51:45.343402 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daac7313-864a-4f82-91a6-0a3bd9099406 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:51:45.344464 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daac7313-864a-4f82-91a6-0a3bd9099406 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:51:45.345053 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daac7313-864a-4f82-91a6-0a3bd9099406 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:51:45.345331 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daac7313-864a-4f82-91a6-0a3bd9099406 None admin] PATH_INFO: `/v3/groups/9b19097b64bb472a99936052ecdf4232` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:51:45.354898 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daac7313-864a-4f82-91a6-0a3bd9099406 None admin] RBAC: Authorizing `identity:get_group(group_id=9b19097b64bb472a99936052ecdf4232)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:51:45.356380 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daac7313-864a-4f82-91a6-0a3bd9099406 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:51:45.360491 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 185/353] 10.222.0.40 () {56 vars in 1180 bytes} [Fri Aug 26 10:51:45 2022] GET /identity/v3/groups/9b19097b64bb472a99936052ecdf4232 => generated 226 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 10:51:45.360491 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daac7313-864a-4f82-91a6-0a3bd9099406 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:51:45.368626 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 169/354] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 26 10:51:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 10:51:45.369423 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b84f976-ec9e-4ea6-9ca5-08614cc51226 None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:51:45.429831 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:51:45.511245 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daac7313-864a-4f82-91a6-0a3bd9099406 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:51:45.513026 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b84f976-ec9e-4ea6-9ca5-08614cc51226 None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:51:45.558897 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5b42b2-de99-4a9d-8e1c-031100719b48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:51:45.559129 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5b42b2-de99-4a9d-8e1c-031100719b48 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:51:45.559368 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5b42b2-de99-4a9d-8e1c-031100719b48 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:51:45.559599 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5b42b2-de99-4a9d-8e1c-031100719b48 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:51:45.562706 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9e2bda-e24b-46ad-b9f9-30e7b0291aed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:51:45.563318 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9e2bda-e24b-46ad-b9f9-30e7b0291aed None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:51:45.563603 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9e2bda-e24b-46ad-b9f9-30e7b0291aed None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:51:45.563869 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9e2bda-e24b-46ad-b9f9-30e7b0291aed None admin] PATH_INFO: `/v3/roles` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:51:45.565151 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e9e2bda-e24b-46ad-b9f9-30e7b0291aed None admin] RBAC: Adding query filter params (name=reader) {{(pid=81423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 10:51:45.565408 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e9e2bda-e24b-46ad-b9f9-30e7b0291aed None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:51:45.566858 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e9e2bda-e24b-46ad-b9f9-30e7b0291aed None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:51:45.577747 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 170/355] 10.222.0.40 () {56 vars in 1101 bytes} [Fri Aug 26 10:51:45 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 10:51:45.579343 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9e2bda-e24b-46ad-b9f9-30e7b0291aed None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:51:45.593445 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:51:45.654728 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f5b42b2-de99-4a9d-8e1c-031100719b48 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:51:45.656319 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f5b42b2-de99-4a9d-8e1c-031100719b48 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:51:45.711679 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9e2bda-e24b-46ad-b9f9-30e7b0291aed None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:51:45.713748 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:51:45.753445 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07bad6ac-e546-4a19-91f9-999aa99e2610 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:51:45.754093 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bad6ac-e546-4a19-91f9-999aa99e2610 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:51:45.754380 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bad6ac-e546-4a19-91f9-999aa99e2610 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:51:45.754653 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bad6ac-e546-4a19-91f9-999aa99e2610 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:51:45.785807 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 186/356] 10.222.0.40 () {58 vars in 1282 bytes} [Fri Aug 26 10:51:45 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 427 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 26 10:51:45.786630 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5b42b2-de99-4a9d-8e1c-031100719b48 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:51:45.842111 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07bad6ac-e546-4a19-91f9-999aa99e2610 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:51:45.843747 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07bad6ac-e546-4a19-91f9-999aa99e2610 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:51:45.854599 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:51:45.914756 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5b42b2-de99-4a9d-8e1c-031100719b48 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:51:45.956847 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d6a039-2985-4f15-8eda-0745fd79c95e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:51:45.957460 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d6a039-2985-4f15-8eda-0745fd79c95e None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:51:45.957731 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d6a039-2985-4f15-8eda-0745fd79c95e None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:51:45.960709 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d6a039-2985-4f15-8eda-0745fd79c95e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:51:45.960709 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d6a039-2985-4f15-8eda-0745fd79c95e None admin] RBAC: Adding query filter params (group.id=9b19097b64bb472a99936052ecdf4232, role.id=23e174ce743b46eabe618c212691da55, scope.project.id=da070dea52de4d71952ed50827d44a1c) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 10:51:45.960709 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d6a039-2985-4f15-8eda-0745fd79c95e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:51:45.960709 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d6a039-2985-4f15-8eda-0745fd79c95e None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:51:45.963511 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 171/357] 10.222.0.40 () {58 vars in 1282 bytes} [Fri Aug 26 10:51:45 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 385 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 26 10:51:45.964761 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc45672-ab9b-4bc6-b13f-26d2cc8ee1d2 None None] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:51:45.965045 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc45672-ab9b-4bc6-b13f-26d2cc8ee1d2 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:51:45.965313 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc45672-ab9b-4bc6-b13f-26d2cc8ee1d2 None None] PATH_INFO: `/` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:51:45.966713 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 172/358] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 26 10:51:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 10:51:45.967254 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc45672-ab9b-4bc6-b13f-26d2cc8ee1d2 None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:51:45.977311 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 187/359] 10.222.0.40 () {56 vars in 1376 bytes} [Fri Aug 26 10:51:45 2022] GET /identity/v3/role_assignments?group.id=9b19097b64bb472a99936052ecdf4232&scope.project.id=da070dea52de4d71952ed50827d44a1c&role.id=23e174ce743b46eabe618c212691da55 => generated 260 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 10:51:45.978790 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d6a039-2985-4f15-8eda-0745fd79c95e None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:51:45.993804 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:51:46.108108 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc45672-ab9b-4bc6-b13f-26d2cc8ee1d2 None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:51:46.113961 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d6a039-2985-4f15-8eda-0745fd79c95e None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:51:46.118306 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:51:46.146557 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748ca44f-f48e-49e4-be9b-5ad90c0f3c73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:51:46.147196 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748ca44f-f48e-49e4-be9b-5ad90c0f3c73 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:51:46.147473 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748ca44f-f48e-49e4-be9b-5ad90c0f3c73 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:51:46.147740 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748ca44f-f48e-49e4-be9b-5ad90c0f3c73 None admin] PATH_INFO: `/v3/users/394cc35228574b0fac9f364cb5fba7f0` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:51:46.149775 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-748ca44f-f48e-49e4-be9b-5ad90c0f3c73 None admin] RBAC: Authorizing `identity:get_user(user_id=394cc35228574b0fac9f364cb5fba7f0)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:51:46.151038 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-748ca44f-f48e-49e4-be9b-5ad90c0f3c73 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:51:46.153568 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 173/360] 10.222.0.40 () {56 vars in 1177 bytes} [Fri Aug 26 10:51:45 2022] GET /identity/v3/users/394cc35228574b0fac9f364cb5fba7f0 => generated 293 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 10:51:46.154438 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748ca44f-f48e-49e4-be9b-5ad90c0f3c73 None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:51:46.161613 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f36536-5dd2-4bb1-a964-27d68dfc04b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:51:46.162292 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f36536-5dd2-4bb1-a964-27d68dfc04b8 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:51:46.162573 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f36536-5dd2-4bb1-a964-27d68dfc04b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:51:46.162843 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f36536-5dd2-4bb1-a964-27d68dfc04b8 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:51:46.175495 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f36536-5dd2-4bb1-a964-27d68dfc04b8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 10:51:46.180832 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application [None req-b2f36536-5dd2-4bb1-a964-27d68dfc04b8 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 26 10:51:46.180832 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 10:51:46.180832 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 10:51:46.180832 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 10:51:46.180832 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 10:51:46.180832 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 10:51:46.180832 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 10:51:46.180832 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 10:51:46.180832 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 10:51:46.180832 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 10:51:46.180832 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 10:51:46.180832 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 10:51:46.180832 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 10:51:46.180832 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 10:51:46.180832 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 10:51:46.180832 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 10:51:46.181707 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 10:51:46.181707 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 10:51:46.181707 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 10:51:46.181707 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 10:51:46.181707 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 10:51:46.181707 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 10:51:46.181707 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 10:51:46.181707 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 10:51:46.181707 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 10:51:46.181707 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 10:51:46.181707 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 10:51:46.181707 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application with Lock( Aug 26 10:51:46.181707 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 10:51:46.181707 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._enter() Aug 26 10:51:46.181707 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 10:51:46.181707 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 10:51:46.182953 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 10:51:46.182953 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.creator() Aug 26 10:51:46.182953 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 10:51:46.182953 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application created_value = creator( Aug 26 10:51:46.182953 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 10:51:46.182953 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 10:51:46.182953 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 10:51:46.182953 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 10:51:46.182953 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 10:51:46.182953 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 10:51:46.182953 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 26 10:51:46.182953 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Aug 26 10:51:46.182953 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 188/361] 10.222.0.40 () {56 vars in 1114 bytes} [Fri Aug 26 10:51:45 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 10:51:46.183617 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa83c3d-140f-4867-9f0c-26f00054b33a None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:51:46.183617 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa83c3d-140f-4867-9f0c-26f00054b33a None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:51:46.183617 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa83c3d-140f-4867-9f0c-26f00054b33a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:51:46.203983 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-5fa83c3d-140f-4867-9f0c-26f00054b33a None None] MFA Rules not processed for user `ca155a48e2024c32af3fd2e19e5fac11`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 10:51:46.205754 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-5fa83c3d-140f-4867-9f0c-26f00054b33a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 10:51:46.286139 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:51:46.291778 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748ca44f-f48e-49e4-be9b-5ad90c0f3c73 None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:51:46.296153 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 189/362] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 26 10:51:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 10:51:46.297023 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa83c3d-140f-4867-9f0c-26f00054b33a None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:51:46.342577 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2cd10e-b7b5-4e8a-92c8-8f3076e5b623 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:51:46.344029 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2cd10e-b7b5-4e8a-92c8-8f3076e5b623 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:51:46.345585 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2cd10e-b7b5-4e8a-92c8-8f3076e5b623 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:51:46.346169 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2cd10e-b7b5-4e8a-92c8-8f3076e5b623 None admin] PATH_INFO: `/v3/projects/15667f91e7f6473cb9aab41830d80f5e` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:51:46.347620 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e2cd10e-b7b5-4e8a-92c8-8f3076e5b623 None admin] RBAC: Authorizing `identity:get_project(project_id=15667f91e7f6473cb9aab41830d80f5e)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:51:46.349237 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e2cd10e-b7b5-4e8a-92c8-8f3076e5b623 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:51:46.352836 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 174/363] 10.222.0.40 () {56 vars in 1186 bytes} [Fri Aug 26 10:51:46 2022] GET /identity/v3/projects/15667f91e7f6473cb9aab41830d80f5e => generated 302 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 10:51:46.358094 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2cd10e-b7b5-4e8a-92c8-8f3076e5b623 None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:51:46.422646 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:51:46.430460 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa83c3d-140f-4867-9f0c-26f00054b33a None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:51:46.472833 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d33a3b9-cf79-4ce0-9363-1b0e5423cd60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:51:46.473907 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d33a3b9-cf79-4ce0-9363-1b0e5423cd60 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:51:46.474454 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d33a3b9-cf79-4ce0-9363-1b0e5423cd60 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:51:46.474936 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d33a3b9-cf79-4ce0-9363-1b0e5423cd60 None admin] PATH_INFO: `/v3/projects/da070dea52de4d71952ed50827d44a1c` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:51:46.476348 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d33a3b9-cf79-4ce0-9363-1b0e5423cd60 None admin] RBAC: Authorizing `identity:get_project(project_id=da070dea52de4d71952ed50827d44a1c)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:51:46.478105 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d33a3b9-cf79-4ce0-9363-1b0e5423cd60 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:51:46.480412 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 190/364] 10.222.0.40 () {56 vars in 1186 bytes} [Fri Aug 26 10:51:46 2022] GET /identity/v3/projects/da070dea52de4d71952ed50827d44a1c => generated 298 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 10:51:46.481562 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d33a3b9-cf79-4ce0-9363-1b0e5423cd60 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:51:46.500673 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2cd10e-b7b5-4e8a-92c8-8f3076e5b623 None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:51:46.547209 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb7051b-0bd1-495f-8d60-4c2fb14c5ed3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:51:46.548205 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb7051b-0bd1-495f-8d60-4c2fb14c5ed3 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:51:46.549398 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb7051b-0bd1-495f-8d60-4c2fb14c5ed3 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:51:46.549872 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb7051b-0bd1-495f-8d60-4c2fb14c5ed3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:51:46.550859 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb7051b-0bd1-495f-8d60-4c2fb14c5ed3 None admin] RBAC: Adding query filter params (role.id=dae11e55a3f343ca9954bc6a1e748eed, user.id=394cc35228574b0fac9f364cb5fba7f0, scope.system=all) {{(pid=81423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 10:51:46.551361 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb7051b-0bd1-495f-8d60-4c2fb14c5ed3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:51:46.552989 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb7051b-0bd1-495f-8d60-4c2fb14c5ed3 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:51:46.557448 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:51:46.566221 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 175/365] 10.222.0.40 () {56 vars in 1308 bytes} [Fri Aug 26 10:51:46 2022] GET /identity/v3/role_assignments?user.id=394cc35228574b0fac9f364cb5fba7f0&scope.system=all&role.id=dae11e55a3f343ca9954bc6a1e748eed => generated 510 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 10:51:46.567500 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb7051b-0bd1-495f-8d60-4c2fb14c5ed3 None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:51:46.631755 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d33a3b9-cf79-4ce0-9363-1b0e5423cd60 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:51:46.686594 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39de6c98-ebfe-4b4e-ad4a-e090fef786a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:51:46.687711 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39de6c98-ebfe-4b4e-ad4a-e090fef786a8 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:51:46.688556 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39de6c98-ebfe-4b4e-ad4a-e090fef786a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:51:46.689165 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39de6c98-ebfe-4b4e-ad4a-e090fef786a8 None admin] PATH_INFO: `/v3/roles` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:51:46.692999 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39de6c98-ebfe-4b4e-ad4a-e090fef786a8 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 10:51:46.693570 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39de6c98-ebfe-4b4e-ad4a-e090fef786a8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:51:46.696063 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39de6c98-ebfe-4b4e-ad4a-e090fef786a8 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:51:46.697419 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb7051b-0bd1-495f-8d60-4c2fb14c5ed3 None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:51:46.700035 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:51:46.715246 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 191/366] 10.222.0.40 () {56 vars in 1111 bytes} [Fri Aug 26 10:51:46 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 10:51:46.715246 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39de6c98-ebfe-4b4e-ad4a-e090fef786a8 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:51:46.751695 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721669c6-2908-49d0-bb42-afe9b98d7352 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:51:46.753265 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721669c6-2908-49d0-bb42-afe9b98d7352 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:51:46.753725 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721669c6-2908-49d0-bb42-afe9b98d7352 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:51:46.754197 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721669c6-2908-49d0-bb42-afe9b98d7352 None admin] PATH_INFO: `/v3/groups/9b19097b64bb472a99936052ecdf4232` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:51:46.756700 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-721669c6-2908-49d0-bb42-afe9b98d7352 None admin] RBAC: Authorizing `identity:get_group(group_id=9b19097b64bb472a99936052ecdf4232)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:51:46.758599 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-721669c6-2908-49d0-bb42-afe9b98d7352 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:51:46.765780 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 176/367] 10.222.0.40 () {56 vars in 1180 bytes} [Fri Aug 26 10:51:46 2022] GET /identity/v3/groups/9b19097b64bb472a99936052ecdf4232 => generated 226 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 10:51:46.767141 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721669c6-2908-49d0-bb42-afe9b98d7352 None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:51:46.843055 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39de6c98-ebfe-4b4e-ad4a-e090fef786a8 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:51:46.885280 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:51:46.900190 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4873272b-8242-4a2f-9d56-f7765d86236a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:51:46.900810 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721669c6-2908-49d0-bb42-afe9b98d7352 None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:51:46.900920 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4873272b-8242-4a2f-9d56-f7765d86236a None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:51:46.901099 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4873272b-8242-4a2f-9d56-f7765d86236a None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:51:46.901373 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4873272b-8242-4a2f-9d56-f7765d86236a None admin] PATH_INFO: `/v3/groups/9b19097b64bb472a99936052ecdf4232` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:51:46.902578 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4873272b-8242-4a2f-9d56-f7765d86236a None admin] RBAC: Authorizing `identity:get_group(group_id=9b19097b64bb472a99936052ecdf4232)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:51:46.904029 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4873272b-8242-4a2f-9d56-f7765d86236a None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:51:46.905929 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 192/368] 10.222.0.40 () {56 vars in 1180 bytes} [Fri Aug 26 10:51:46 2022] GET /identity/v3/groups/9b19097b64bb472a99936052ecdf4232 => generated 226 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 10:51:46.909671 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4873272b-8242-4a2f-9d56-f7765d86236a None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:51:46.938600 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f33655e-8210-423a-a1b0-bb5beb00db63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:51:46.939343 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f33655e-8210-423a-a1b0-bb5beb00db63 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:51:46.940131 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f33655e-8210-423a-a1b0-bb5beb00db63 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:51:46.940398 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f33655e-8210-423a-a1b0-bb5beb00db63 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:51:46.959648 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f33655e-8210-423a-a1b0-bb5beb00db63 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 10:51:46.965110 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application [None req-4f33655e-8210-423a-a1b0-bb5beb00db63 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 26 10:51:46.965110 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 10:51:46.965110 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 10:51:46.965110 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 10:51:46.965110 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 10:51:46.965110 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 10:51:46.965110 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 10:51:46.965110 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 10:51:46.965110 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 10:51:46.965110 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 10:51:46.965110 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 10:51:46.965110 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 10:51:46.965110 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 10:51:46.965110 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 10:51:46.965110 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 10:51:46.965110 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 10:51:46.966063 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 10:51:46.966063 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 10:51:46.966063 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 10:51:46.966063 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 10:51:46.966063 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 10:51:46.966063 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 10:51:46.966063 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 10:51:46.966063 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 10:51:46.966063 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 10:51:46.966063 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 10:51:46.966063 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 10:51:46.966063 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application with Lock( Aug 26 10:51:46.966063 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 10:51:46.966063 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._enter() Aug 26 10:51:46.966063 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 10:51:46.966063 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 10:51:46.966894 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 10:51:46.966894 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.creator() Aug 26 10:51:46.966894 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 10:51:46.966894 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application created_value = creator( Aug 26 10:51:46.966894 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 10:51:46.966894 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 10:51:46.966894 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 10:51:46.966894 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 10:51:46.966894 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 10:51:46.966894 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 10:51:46.966894 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 26 10:51:46.966894 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Aug 26 10:51:46.966894 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 177/369] 10.222.0.40 () {56 vars in 1096 bytes} [Fri Aug 26 10:51:46 2022] GET /identity/v3/roles/admin => generated 83 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 10:51:46.967542 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f33655e-8210-423a-a1b0-bb5beb00db63 None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:51:47.007654 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:51:47.041125 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4873272b-8242-4a2f-9d56-f7765d86236a None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:51:47.089397 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7abe96a8-4297-46a7-9a09-29de5d2b9a05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:51:47.090048 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abe96a8-4297-46a7-9a09-29de5d2b9a05 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:51:47.090330 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abe96a8-4297-46a7-9a09-29de5d2b9a05 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:51:47.090600 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abe96a8-4297-46a7-9a09-29de5d2b9a05 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:51:47.101857 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f33655e-8210-423a-a1b0-bb5beb00db63 None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:51:47.143317 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ccacd4c-6107-4756-92af-39a8147094e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:51:47.145145 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ccacd4c-6107-4756-92af-39a8147094e3 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:51:47.145590 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ccacd4c-6107-4756-92af-39a8147094e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:51:47.146391 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ccacd4c-6107-4756-92af-39a8147094e3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:51:47.147276 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ccacd4c-6107-4756-92af-39a8147094e3 None admin] RBAC: Adding query filter params (group.id=9b19097b64bb472a99936052ecdf4232, role.id=23e174ce743b46eabe618c212691da55, scope.project.id=15667f91e7f6473cb9aab41830d80f5e) {{(pid=81423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 10:51:47.147472 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ccacd4c-6107-4756-92af-39a8147094e3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:51:47.149007 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ccacd4c-6107-4756-92af-39a8147094e3 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:51:47.155298 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 178/370] 10.222.0.40 () {56 vars in 1376 bytes} [Fri Aug 26 10:51:46 2022] GET /identity/v3/role_assignments?group.id=9b19097b64bb472a99936052ecdf4232&scope.project.id=15667f91e7f6473cb9aab41830d80f5e&role.id=23e174ce743b46eabe618c212691da55 => generated 260 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 10:51:47.162019 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ccacd4c-6107-4756-92af-39a8147094e3 None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:51:47.172940 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:51:47.184787 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7abe96a8-4297-46a7-9a09-29de5d2b9a05 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:51:47.186707 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7abe96a8-4297-46a7-9a09-29de5d2b9a05 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:51:47.319073 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:51:47.330848 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ccacd4c-6107-4756-92af-39a8147094e3 None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:51:47.333923 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 193/371] 10.222.0.40 () {58 vars in 1282 bytes} [Fri Aug 26 10:51:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 426 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 26 10:51:47.334822 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7abe96a8-4297-46a7-9a09-29de5d2b9a05 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:51:47.376569 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac80a6d-c901-4301-8ce3-c53227dfcfb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:51:47.378263 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac80a6d-c901-4301-8ce3-c53227dfcfb2 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:51:47.378746 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac80a6d-c901-4301-8ce3-c53227dfcfb2 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:51:47.379217 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac80a6d-c901-4301-8ce3-c53227dfcfb2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:51:47.380987 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac80a6d-c901-4301-8ce3-c53227dfcfb2 None admin] RBAC: Adding query filter params (group.id=9b19097b64bb472a99936052ecdf4232, role.id=9c439f898126444f92abb01910f1d02b, scope.project.id=da070dea52de4d71952ed50827d44a1c) {{(pid=81423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 10:51:47.381529 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac80a6d-c901-4301-8ce3-c53227dfcfb2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:51:47.384476 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac80a6d-c901-4301-8ce3-c53227dfcfb2 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:51:47.392007 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 179/372] 10.222.0.40 () {56 vars in 1376 bytes} [Fri Aug 26 10:51:47 2022] GET /identity/v3/role_assignments?group.id=9b19097b64bb472a99936052ecdf4232&scope.project.id=da070dea52de4d71952ed50827d44a1c&role.id=9c439f898126444f92abb01910f1d02b => generated 260 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 10:51:47.393555 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac80a6d-c901-4301-8ce3-c53227dfcfb2 None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:51:47.451024 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:51:47.460437 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7abe96a8-4297-46a7-9a09-29de5d2b9a05 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:51:47.507787 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8d5256-ed53-42eb-a375-c4eac3fc12c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:51:47.509449 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8d5256-ed53-42eb-a375-c4eac3fc12c5 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:51:47.509726 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8d5256-ed53-42eb-a375-c4eac3fc12c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:51:47.510066 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8d5256-ed53-42eb-a375-c4eac3fc12c5 None admin] PATH_INFO: `/v3/roles` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:51:47.510745 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f8d5256-ed53-42eb-a375-c4eac3fc12c5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 10:51:47.511042 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f8d5256-ed53-42eb-a375-c4eac3fc12c5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:51:47.512446 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f8d5256-ed53-42eb-a375-c4eac3fc12c5 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:51:47.523463 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 194/373] 10.222.0.40 () {56 vars in 1099 bytes} [Fri Aug 26 10:51:47 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 10:51:47.528969 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8d5256-ed53-42eb-a375-c4eac3fc12c5 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:51:47.562239 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac80a6d-c901-4301-8ce3-c53227dfcfb2 None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:51:47.631922 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2a1b58-4e06-4bdb-ab68-14c6159c480b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:51:47.633070 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2a1b58-4e06-4bdb-ab68-14c6159c480b None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:51:47.634670 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2a1b58-4e06-4bdb-ab68-14c6159c480b None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:51:47.634960 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2a1b58-4e06-4bdb-ab68-14c6159c480b None admin] PATH_INFO: `/v3/projects/15667f91e7f6473cb9aab41830d80f5e` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:51:47.636167 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba2a1b58-4e06-4bdb-ab68-14c6159c480b None admin] RBAC: Authorizing `identity:get_project(project_id=15667f91e7f6473cb9aab41830d80f5e)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:51:47.637619 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba2a1b58-4e06-4bdb-ab68-14c6159c480b None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:51:47.645028 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 180/374] 10.222.0.40 () {56 vars in 1186 bytes} [Fri Aug 26 10:51:47 2022] GET /identity/v3/projects/15667f91e7f6473cb9aab41830d80f5e => generated 302 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 10:51:47.645959 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2a1b58-4e06-4bdb-ab68-14c6159c480b None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:51:47.683419 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:51:47.707228 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8d5256-ed53-42eb-a375-c4eac3fc12c5 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:51:47.747531 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26eacf4-5a18-4f58-81ac-24a356fb3ebe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:51:47.748168 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26eacf4-5a18-4f58-81ac-24a356fb3ebe None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:51:47.748447 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26eacf4-5a18-4f58-81ac-24a356fb3ebe None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:51:47.748721 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26eacf4-5a18-4f58-81ac-24a356fb3ebe None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:51:47.775141 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:51:47.805779 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2a1b58-4e06-4bdb-ab68-14c6159c480b None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:51:47.837696 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f26eacf4-5a18-4f58-81ac-24a356fb3ebe None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:51:47.839195 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f26eacf4-5a18-4f58-81ac-24a356fb3ebe None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:51:47.851597 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad35e6f5-22e6-4dfd-affb-f3799075ab9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:51:47.852609 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad35e6f5-22e6-4dfd-affb-f3799075ab9b None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:51:47.853102 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad35e6f5-22e6-4dfd-affb-f3799075ab9b None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:51:47.853576 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad35e6f5-22e6-4dfd-affb-f3799075ab9b None admin] PATH_INFO: `/v3/groups/9b19097b64bb472a99936052ecdf4232` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:51:47.855167 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad35e6f5-22e6-4dfd-affb-f3799075ab9b None admin] RBAC: Authorizing `identity:get_group(group_id=9b19097b64bb472a99936052ecdf4232)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:51:47.856973 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad35e6f5-22e6-4dfd-affb-f3799075ab9b None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:51:47.864190 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 181/375] 10.222.0.40 () {56 vars in 1180 bytes} [Fri Aug 26 10:51:47 2022] GET /identity/v3/groups/9b19097b64bb472a99936052ecdf4232 => generated 226 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 10:51:47.865840 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad35e6f5-22e6-4dfd-affb-f3799075ab9b None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:51:48.006879 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 195/376] 10.222.0.40 () {58 vars in 1282 bytes} [Fri Aug 26 10:51:47 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 478 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 26 10:51:48.011451 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26eacf4-5a18-4f58-81ac-24a356fb3ebe None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:51:48.065727 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:51:48.077890 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad35e6f5-22e6-4dfd-affb-f3799075ab9b None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:51:48.118186 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae111b9-bed2-410b-9c04-15526c000675 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:51:48.118803 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae111b9-bed2-410b-9c04-15526c000675 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:51:48.119079 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae111b9-bed2-410b-9c04-15526c000675 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:51:48.119348 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae111b9-bed2-410b-9c04-15526c000675 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:51:48.120078 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae111b9-bed2-410b-9c04-15526c000675 None admin] RBAC: Adding query filter params (group.id=9b19097b64bb472a99936052ecdf4232, role.id=9c439f898126444f92abb01910f1d02b, scope.project.id=15667f91e7f6473cb9aab41830d80f5e) {{(pid=81423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 10:51:48.120380 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae111b9-bed2-410b-9c04-15526c000675 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:51:48.121928 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae111b9-bed2-410b-9c04-15526c000675 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:51:48.122816 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:51:48.134003 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 182/377] 10.222.0.40 () {56 vars in 1376 bytes} [Fri Aug 26 10:51:47 2022] GET /identity/v3/role_assignments?group.id=9b19097b64bb472a99936052ecdf4232&scope.project.id=15667f91e7f6473cb9aab41830d80f5e&role.id=9c439f898126444f92abb01910f1d02b => generated 260 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 10:51:48.134003 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88cbb85-61eb-4436-a7eb-45a85ec325d1 None None] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:51:48.134003 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88cbb85-61eb-4436-a7eb-45a85ec325d1 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:51:48.134003 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88cbb85-61eb-4436-a7eb-45a85ec325d1 None None] PATH_INFO: `/` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:51:48.136291 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 183/378] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 26 10:51:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 10:51:48.137597 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87dacbb8-7128-4cb7-94dd-4f755aab2baa None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:51:48.137880 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87dacbb8-7128-4cb7-94dd-4f755aab2baa None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:51:48.138190 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87dacbb8-7128-4cb7-94dd-4f755aab2baa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:51:48.157199 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-87dacbb8-7128-4cb7-94dd-4f755aab2baa None None] MFA Rules not processed for user `ca155a48e2024c32af3fd2e19e5fac11`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 10:51:48.158995 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-87dacbb8-7128-4cb7-94dd-4f755aab2baa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 10:51:48.229389 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26eacf4-5a18-4f58-81ac-24a356fb3ebe None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:51:48.234458 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 184/379] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 26 10:51:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 10:51:48.301329 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34780dac-0a8e-4ceb-9909-719ec983d747 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:51:48.302423 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34780dac-0a8e-4ceb-9909-719ec983d747 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:51:48.304742 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34780dac-0a8e-4ceb-9909-719ec983d747 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:51:48.305227 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34780dac-0a8e-4ceb-9909-719ec983d747 None admin] PATH_INFO: `/v3/projects/f3942b05c97e4a1ca048ce3a0379b7a7` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:51:48.306821 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34780dac-0a8e-4ceb-9909-719ec983d747 None admin] RBAC: Authorizing `identity:get_project(project_id=f3942b05c97e4a1ca048ce3a0379b7a7)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:51:48.308559 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34780dac-0a8e-4ceb-9909-719ec983d747 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:51:48.311020 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 196/380] 10.222.0.40 () {56 vars in 1186 bytes} [Fri Aug 26 10:51:48 2022] GET /identity/v3/projects/f3942b05c97e4a1ca048ce3a0379b7a7 => generated 344 bytes in 301 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 10:51:48.319794 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87dacbb8-7128-4cb7-94dd-4f755aab2baa None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:51:48.426036 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:51:48.438286 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87dacbb8-7128-4cb7-94dd-4f755aab2baa None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:51:48.475019 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68fbb0f-e7d6-4f60-8b9a-6565d2ef314f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:51:48.475655 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68fbb0f-e7d6-4f60-8b9a-6565d2ef314f None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:51:48.475932 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68fbb0f-e7d6-4f60-8b9a-6565d2ef314f None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:51:48.476202 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68fbb0f-e7d6-4f60-8b9a-6565d2ef314f None admin] PATH_INFO: `/v3/groups/907d2ba571f44f6c8cf2a75be1eda1e0` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:51:48.482189 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f68fbb0f-e7d6-4f60-8b9a-6565d2ef314f None admin] RBAC: Authorizing `identity:get_group(group_id=907d2ba571f44f6c8cf2a75be1eda1e0)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:51:48.483877 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f68fbb0f-e7d6-4f60-8b9a-6565d2ef314f None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:51:48.485632 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 185/381] 10.222.0.40 () {56 vars in 1180 bytes} [Fri Aug 26 10:51:48 2022] GET /identity/v3/groups/907d2ba571f44f6c8cf2a75be1eda1e0 => generated 229 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 10:51:48.491373 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34780dac-0a8e-4ceb-9909-719ec983d747 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:51:48.628652 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34780dac-0a8e-4ceb-9909-719ec983d747 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:51:48.650619 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68fbb0f-e7d6-4f60-8b9a-6565d2ef314f None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:51:48.667321 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db307709-5af9-46e8-9316-81b1a08d1096 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:51:48.667943 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db307709-5af9-46e8-9316-81b1a08d1096 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:51:48.668227 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db307709-5af9-46e8-9316-81b1a08d1096 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:51:48.668507 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db307709-5af9-46e8-9316-81b1a08d1096 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:51:48.669234 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db307709-5af9-46e8-9316-81b1a08d1096 None admin] RBAC: Adding query filter params (group.id=907d2ba571f44f6c8cf2a75be1eda1e0, role.id=1f97e4602e2e4d6498913a81f9ae8d6a, scope.project.id=f3942b05c97e4a1ca048ce3a0379b7a7) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 10:51:48.669546 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db307709-5af9-46e8-9316-81b1a08d1096 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:51:48.671094 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db307709-5af9-46e8-9316-81b1a08d1096 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:51:48.674299 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:51:48.677173 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 197/382] 10.222.0.40 () {56 vars in 1376 bytes} [Fri Aug 26 10:51:48 2022] GET /identity/v3/role_assignments?group.id=907d2ba571f44f6c8cf2a75be1eda1e0&scope.project.id=f3942b05c97e4a1ca048ce3a0379b7a7&role.id=1f97e4602e2e4d6498913a81f9ae8d6a => generated 260 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 10:51:48.790448 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68fbb0f-e7d6-4f60-8b9a-6565d2ef314f None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:51:48.826877 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e013a92-5c65-4eb6-834b-2989627f930a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:51:48.827517 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e013a92-5c65-4eb6-834b-2989627f930a None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:51:48.827797 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e013a92-5c65-4eb6-834b-2989627f930a None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:51:48.828070 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e013a92-5c65-4eb6-834b-2989627f930a None admin] PATH_INFO: `/v3/roles/member` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:51:48.840236 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e013a92-5c65-4eb6-834b-2989627f930a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 10:51:48.845836 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application [None req-6e013a92-5c65-4eb6-834b-2989627f930a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 26 10:51:48.845836 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 10:51:48.845836 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 10:51:48.845836 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 10:51:48.845836 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 10:51:48.845836 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 10:51:48.845836 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 10:51:48.845836 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 10:51:48.845836 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 10:51:48.845836 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 10:51:48.845836 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 10:51:48.845836 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 10:51:48.845836 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 10:51:48.845836 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 10:51:48.845836 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 10:51:48.845836 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 10:51:48.846785 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 10:51:48.846785 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 10:51:48.846785 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 10:51:48.846785 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 10:51:48.846785 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 10:51:48.846785 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 10:51:48.846785 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 10:51:48.846785 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 10:51:48.846785 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 10:51:48.846785 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 10:51:48.846785 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 10:51:48.846785 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application with Lock( Aug 26 10:51:48.846785 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 10:51:48.846785 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._enter() Aug 26 10:51:48.846785 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 10:51:48.846785 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 10:51:48.847706 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 10:51:48.847706 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.creator() Aug 26 10:51:48.847706 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 10:51:48.847706 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application created_value = creator( Aug 26 10:51:48.847706 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 10:51:48.847706 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 10:51:48.847706 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 10:51:48.847706 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 10:51:48.847706 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 10:51:48.847706 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 10:51:48.847706 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 26 10:51:48.847706 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Aug 26 10:51:48.847706 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 186/383] 10.222.0.40 () {56 vars in 1099 bytes} [Fri Aug 26 10:51:48 2022] GET /identity/v3/roles/member => generated 84 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 10:51:48.856110 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db307709-5af9-46e8-9316-81b1a08d1096 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:51:48.999852 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db307709-5af9-46e8-9316-81b1a08d1096 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:51:49.038287 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4df868-9b8d-4b95-8d4e-ccfd52b42575 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:51:49.038970 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4df868-9b8d-4b95-8d4e-ccfd52b42575 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:51:49.039248 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4df868-9b8d-4b95-8d4e-ccfd52b42575 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:51:49.039518 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4df868-9b8d-4b95-8d4e-ccfd52b42575 None admin] PATH_INFO: `/v3/roles` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:51:49.040191 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f4df868-9b8d-4b95-8d4e-ccfd52b42575 None admin] RBAC: Adding query filter params (name=member) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 10:51:49.040591 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f4df868-9b8d-4b95-8d4e-ccfd52b42575 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:51:49.041987 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f4df868-9b8d-4b95-8d4e-ccfd52b42575 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:51:49.050942 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 198/384] 10.222.0.40 () {56 vars in 1101 bytes} [Fri Aug 26 10:51:48 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 10:51:49.056334 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e013a92-5c65-4eb6-834b-2989627f930a None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:51:49.076582 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:51:49.175894 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e013a92-5c65-4eb6-834b-2989627f930a None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:51:49.218090 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9d36e7-4bd8-4fb0-ac36-cb3325cf54bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:51:49.218733 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9d36e7-4bd8-4fb0-ac36-cb3325cf54bd None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:51:49.219062 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9d36e7-4bd8-4fb0-ac36-cb3325cf54bd None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:51:49.219342 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9d36e7-4bd8-4fb0-ac36-cb3325cf54bd None admin] PATH_INFO: `/v3/groups/9b19097b64bb472a99936052ecdf4232` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:51:49.220495 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca9d36e7-4bd8-4fb0-ac36-cb3325cf54bd None admin] RBAC: Authorizing `identity:get_group(group_id=9b19097b64bb472a99936052ecdf4232)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:51:49.221949 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca9d36e7-4bd8-4fb0-ac36-cb3325cf54bd None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:51:49.223624 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 187/385] 10.222.0.40 () {56 vars in 1180 bytes} [Fri Aug 26 10:51:49 2022] GET /identity/v3/groups/9b19097b64bb472a99936052ecdf4232 => generated 226 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 10:51:49.229161 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4df868-9b8d-4b95-8d4e-ccfd52b42575 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:51:49.282530 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:51:49.335017 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4df868-9b8d-4b95-8d4e-ccfd52b42575 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:51:49.370594 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848435da-2749-4717-8ce5-4333ff99f05d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:51:49.371312 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848435da-2749-4717-8ce5-4333ff99f05d None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:51:49.371589 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848435da-2749-4717-8ce5-4333ff99f05d None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:51:49.371862 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848435da-2749-4717-8ce5-4333ff99f05d None admin] PATH_INFO: `/v3/projects/da070dea52de4d71952ed50827d44a1c` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:51:49.372985 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-848435da-2749-4717-8ce5-4333ff99f05d None admin] RBAC: Authorizing `identity:get_project(project_id=da070dea52de4d71952ed50827d44a1c)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:51:49.374526 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-848435da-2749-4717-8ce5-4333ff99f05d None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:51:49.376463 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 199/386] 10.222.0.40 () {56 vars in 1186 bytes} [Fri Aug 26 10:51:49 2022] GET /identity/v3/projects/da070dea52de4d71952ed50827d44a1c => generated 298 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 10:51:49.383060 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9d36e7-4bd8-4fb0-ac36-cb3325cf54bd None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:51:49.496066 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:51:49.528460 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9d36e7-4bd8-4fb0-ac36-cb3325cf54bd None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:51:49.583663 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd06cd1-4b6b-4435-939d-3cb30ccc3080 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:51:49.584679 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd06cd1-4b6b-4435-939d-3cb30ccc3080 None admin] REQUEST_METHOD: `PUT` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:51:49.585159 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd06cd1-4b6b-4435-939d-3cb30ccc3080 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:51:49.585616 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd06cd1-4b6b-4435-939d-3cb30ccc3080 None admin] PATH_INFO: `/v3/projects/da070dea52de4d71952ed50827d44a1c/groups/9b19097b64bb472a99936052ecdf4232/roles/23e174ce743b46eabe618c212691da55` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:51:49.590301 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd06cd1-4b6b-4435-939d-3cb30ccc3080 None admin] RBAC: Authorizing `identity:create_grant(project_id=da070dea52de4d71952ed50827d44a1c, group_id=9b19097b64bb472a99936052ecdf4232, role_id=23e174ce743b46eabe618c212691da55)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:51:49.591271 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a5b499-e73f-4c47-b701-dc9f34f7e118 None None] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:51:49.591554 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a5b499-e73f-4c47-b701-dc9f34f7e118 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:51:49.591748 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd06cd1-4b6b-4435-939d-3cb30ccc3080 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:51:49.591844 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a5b499-e73f-4c47-b701-dc9f34f7e118 None None] PATH_INFO: `/` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:51:49.593275 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 200/387] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 26 10:51:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 10:51:49.601021 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bcede70-2010-4086-9b93-305e4e6c72a2 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:51:49.601306 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bcede70-2010-4086-9b93-305e4e6c72a2 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:51:49.601581 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bcede70-2010-4086-9b93-305e4e6c72a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:51:49.608296 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 188/388] 10.222.0.40 () {58 vars in 1442 bytes} [Fri Aug 26 10:51:49 2022] PUT /identity/v3/projects/da070dea52de4d71952ed50827d44a1c/groups/9b19097b64bb472a99936052ecdf4232/roles/23e174ce743b46eabe618c212691da55 => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 10:51:49.633431 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-6bcede70-2010-4086-9b93-305e4e6c72a2 None None] MFA Rules not processed for user `ca155a48e2024c32af3fd2e19e5fac11`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 10:51:49.634699 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-6bcede70-2010-4086-9b93-305e4e6c72a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 10:51:49.739672 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 201/389] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 26 10:51:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 10:51:50.119056 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd06cd1-4b6b-4435-939d-3cb30ccc3080 None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:51:50.238146 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd06cd1-4b6b-4435-939d-3cb30ccc3080 None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:51:50.285725 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a9fea0-32ba-4ccf-9451-21f0d85b9328 None None] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:51:50.286244 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3781482-dd85-4841-8315-e39cff7d3af9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:51:50.286340 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a9fea0-32ba-4ccf-9451-21f0d85b9328 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:51:50.286340 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a9fea0-32ba-4ccf-9451-21f0d85b9328 None None] PATH_INFO: `/` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:51:50.286502 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3781482-dd85-4841-8315-e39cff7d3af9 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:51:50.286768 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3781482-dd85-4841-8315-e39cff7d3af9 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:51:50.287014 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3781482-dd85-4841-8315-e39cff7d3af9 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:51:50.287848 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 202/390] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 26 10:51:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 10:51:50.297363 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca9d729-7c44-4cde-a539-d041a2d94918 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:51:50.297646 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca9d729-7c44-4cde-a539-d041a2d94918 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:51:50.297921 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca9d729-7c44-4cde-a539-d041a2d94918 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:51:50.298334 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:51:50.301603 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3781482-dd85-4841-8315-e39cff7d3af9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 10:51:50.307303 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application [None req-c3781482-dd85-4841-8315-e39cff7d3af9 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 26 10:51:50.307303 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 10:51:50.307303 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 10:51:50.307303 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 10:51:50.307303 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 10:51:50.307303 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 10:51:50.307303 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 10:51:50.307303 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 10:51:50.307303 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 10:51:50.307303 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 10:51:50.307303 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 10:51:50.307303 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 10:51:50.307303 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 10:51:50.307303 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 10:51:50.307303 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 10:51:50.307303 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 10:51:50.308188 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 10:51:50.308188 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 10:51:50.308188 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 10:51:50.308188 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 10:51:50.308188 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 10:51:50.308188 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 10:51:50.308188 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 10:51:50.308188 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 10:51:50.308188 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 10:51:50.308188 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 10:51:50.308188 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 10:51:50.308188 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application with Lock( Aug 26 10:51:50.308188 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 10:51:50.308188 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._enter() Aug 26 10:51:50.308188 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 10:51:50.308188 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 10:51:50.309487 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 10:51:50.309487 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.creator() Aug 26 10:51:50.309487 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 10:51:50.309487 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application created_value = creator( Aug 26 10:51:50.309487 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 10:51:50.309487 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 10:51:50.309487 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 10:51:50.309487 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 10:51:50.309487 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 10:51:50.309487 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 10:51:50.309487 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 26 10:51:50.309487 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Aug 26 10:51:50.309487 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 189/391] 10.222.0.40 () {56 vars in 1099 bytes} [Fri Aug 26 10:51:50 2022] GET /identity/v3/roles/member => generated 84 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 10:51:50.320302 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3781482-dd85-4841-8315-e39cff7d3af9 None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:51:50.338400 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-5ca9d729-7c44-4cde-a539-d041a2d94918 None None] MFA Rules not processed for user `ca155a48e2024c32af3fd2e19e5fac11`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 10:51:50.339605 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-5ca9d729-7c44-4cde-a539-d041a2d94918 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 10:51:50.429740 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:51:50.432653 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3781482-dd85-4841-8315-e39cff7d3af9 None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:51:50.469416 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752bf717-e8f6-4c53-ac02-a9e384f04b37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:51:50.470052 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752bf717-e8f6-4c53-ac02-a9e384f04b37 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:51:50.470329 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752bf717-e8f6-4c53-ac02-a9e384f04b37 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:51:50.471137 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752bf717-e8f6-4c53-ac02-a9e384f04b37 None admin] PATH_INFO: `/v3/roles` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:51:50.471802 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-752bf717-e8f6-4c53-ac02-a9e384f04b37 None admin] RBAC: Adding query filter params (name=member) {{(pid=81423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 10:51:50.472105 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-752bf717-e8f6-4c53-ac02-a9e384f04b37 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:51:50.473561 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-752bf717-e8f6-4c53-ac02-a9e384f04b37 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:51:50.480957 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 203/392] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 26 10:51:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 10:51:50.482259 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e17d56-1617-4a48-bda1-a36f7f6203d8 None None] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:51:50.482541 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e17d56-1617-4a48-bda1-a36f7f6203d8 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:51:50.483062 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e17d56-1617-4a48-bda1-a36f7f6203d8 None None] PATH_INFO: `/` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:51:50.484614 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 204/393] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 26 10:51:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 10:51:50.489466 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 190/394] 10.222.0.40 () {56 vars in 1101 bytes} [Fri Aug 26 10:51:50 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 10:51:50.494084 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60699529-7b55-48ab-8d9e-127e15f07fd2 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:51:50.494084 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60699529-7b55-48ab-8d9e-127e15f07fd2 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:51:50.494084 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60699529-7b55-48ab-8d9e-127e15f07fd2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:51:50.495386 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752bf717-e8f6-4c53-ac02-a9e384f04b37 None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:51:50.518531 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-60699529-7b55-48ab-8d9e-127e15f07fd2 None None] MFA Rules not processed for user `ca155a48e2024c32af3fd2e19e5fac11`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 10:51:50.519997 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-60699529-7b55-48ab-8d9e-127e15f07fd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 10:51:50.622882 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 205/395] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 26 10:51:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 10:51:50.638627 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:51:50.690023 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752bf717-e8f6-4c53-ac02-a9e384f04b37 None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:51:50.735519 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41138bef-7f31-4e22-a7e4-511ee8f331e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:51:50.737341 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41138bef-7f31-4e22-a7e4-511ee8f331e0 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:51:50.737820 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41138bef-7f31-4e22-a7e4-511ee8f331e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:51:50.738353 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41138bef-7f31-4e22-a7e4-511ee8f331e0 None admin] PATH_INFO: `/v3/groups/9b19097b64bb472a99936052ecdf4232` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:51:50.739799 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41138bef-7f31-4e22-a7e4-511ee8f331e0 None admin] RBAC: Authorizing `identity:get_group(group_id=9b19097b64bb472a99936052ecdf4232)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:51:50.741465 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41138bef-7f31-4e22-a7e4-511ee8f331e0 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:51:50.743249 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 191/396] 10.222.0.40 () {56 vars in 1180 bytes} [Fri Aug 26 10:51:50 2022] GET /identity/v3/groups/9b19097b64bb472a99936052ecdf4232 => generated 226 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 10:51:50.748734 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60699529-7b55-48ab-8d9e-127e15f07fd2 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:51:50.817732 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41138bef-7f31-4e22-a7e4-511ee8f331e0 None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:51:50.856201 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60699529-7b55-48ab-8d9e-127e15f07fd2 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:51:50.866408 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:51:50.901919 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74243582-8534-4e7c-94f8-4c9e2414b96c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:51:50.902598 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74243582-8534-4e7c-94f8-4c9e2414b96c None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:51:50.902865 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74243582-8534-4e7c-94f8-4c9e2414b96c None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:51:50.903137 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74243582-8534-4e7c-94f8-4c9e2414b96c None admin] PATH_INFO: `/v3/projects/15667f91e7f6473cb9aab41830d80f5e` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:51:50.906708 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74243582-8534-4e7c-94f8-4c9e2414b96c None admin] RBAC: Authorizing `identity:get_project(project_id=15667f91e7f6473cb9aab41830d80f5e)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:51:50.908322 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74243582-8534-4e7c-94f8-4c9e2414b96c None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:51:50.914820 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 206/397] 10.222.0.40 () {56 vars in 1186 bytes} [Fri Aug 26 10:51:50 2022] GET /identity/v3/projects/15667f91e7f6473cb9aab41830d80f5e => generated 302 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 10:51:50.920167 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74243582-8534-4e7c-94f8-4c9e2414b96c None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:51:50.939980 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41138bef-7f31-4e22-a7e4-511ee8f331e0 None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:51:50.971531 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:51:50.977809 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee92e40-89c2-4f5f-944a-65a6dfcc0443 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:51:50.978809 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee92e40-89c2-4f5f-944a-65a6dfcc0443 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:51:50.979441 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee92e40-89c2-4f5f-944a-65a6dfcc0443 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:51:50.979912 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee92e40-89c2-4f5f-944a-65a6dfcc0443 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:51:50.992727 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dee92e40-89c2-4f5f-944a-65a6dfcc0443 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 10:51:50.998566 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application [None req-dee92e40-89c2-4f5f-944a-65a6dfcc0443 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 26 10:51:50.998566 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 10:51:50.998566 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 10:51:50.998566 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 10:51:50.998566 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 10:51:50.998566 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 10:51:50.998566 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 10:51:50.998566 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 10:51:50.998566 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 10:51:50.998566 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 10:51:50.998566 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 10:51:50.998566 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 10:51:50.998566 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 10:51:50.998566 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 10:51:50.998566 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 10:51:50.998566 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 10:51:50.999537 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 10:51:50.999537 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 10:51:50.999537 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 10:51:50.999537 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 10:51:50.999537 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 10:51:50.999537 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 10:51:50.999537 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 10:51:50.999537 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 10:51:50.999537 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 10:51:50.999537 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 10:51:50.999537 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 10:51:50.999537 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application with Lock( Aug 26 10:51:50.999537 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 10:51:50.999537 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._enter() Aug 26 10:51:50.999537 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 10:51:50.999537 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 10:51:51.000310 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 10:51:51.000310 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.creator() Aug 26 10:51:51.000310 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 10:51:51.000310 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application created_value = creator( Aug 26 10:51:51.000310 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 10:51:51.000310 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 10:51:51.000310 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 10:51:51.000310 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 10:51:51.000310 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 10:51:51.000310 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 10:51:51.000310 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 26 10:51:51.000310 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Aug 26 10:51:51.003749 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 192/398] 10.222.0.40 () {56 vars in 1114 bytes} [Fri Aug 26 10:51:50 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 26 10:51:51.008971 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee92e40-89c2-4f5f-944a-65a6dfcc0443 None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:51:51.083749 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74243582-8534-4e7c-94f8-4c9e2414b96c None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:51:51.123414 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0c3fae-4b67-4e2b-8556-16a45f0af291 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:51:51.124092 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0c3fae-4b67-4e2b-8556-16a45f0af291 None admin] REQUEST_METHOD: `PUT` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:51:51.124335 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0c3fae-4b67-4e2b-8556-16a45f0af291 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:51:51.124604 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0c3fae-4b67-4e2b-8556-16a45f0af291 None admin] PATH_INFO: `/v3/projects/15667f91e7f6473cb9aab41830d80f5e/groups/9b19097b64bb472a99936052ecdf4232/roles/23e174ce743b46eabe618c212691da55` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:51:51.126597 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c0c3fae-4b67-4e2b-8556-16a45f0af291 None admin] RBAC: Authorizing `identity:create_grant(project_id=15667f91e7f6473cb9aab41830d80f5e, group_id=9b19097b64bb472a99936052ecdf4232, role_id=23e174ce743b46eabe618c212691da55)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:51:51.128115 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c0c3fae-4b67-4e2b-8556-16a45f0af291 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:51:51.143484 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 207/399] 10.222.0.40 () {58 vars in 1442 bytes} [Fri Aug 26 10:51:50 2022] PUT /identity/v3/projects/15667f91e7f6473cb9aab41830d80f5e/groups/9b19097b64bb472a99936052ecdf4232/roles/23e174ce743b46eabe618c212691da55 => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 26 10:51:51.144512 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0c3fae-4b67-4e2b-8556-16a45f0af291 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:51:51.150238 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee92e40-89c2-4f5f-944a-65a6dfcc0443 None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:51:51.190573 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f9bbc71-52a2-4648-9464-ff4e7fa9b222 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:51:51.191607 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9bbc71-52a2-4648-9464-ff4e7fa9b222 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:51:51.192088 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9bbc71-52a2-4648-9464-ff4e7fa9b222 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:51:51.192584 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9bbc71-52a2-4648-9464-ff4e7fa9b222 None admin] PATH_INFO: `/v3/roles` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:51:51.193491 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f9bbc71-52a2-4648-9464-ff4e7fa9b222 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 10:51:51.194041 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f9bbc71-52a2-4648-9464-ff4e7fa9b222 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:51:51.195671 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f9bbc71-52a2-4648-9464-ff4e7fa9b222 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:51:51.205675 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 193/400] 10.222.0.40 () {56 vars in 1111 bytes} [Fri Aug 26 10:51:51 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 10:51:51.207448 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc557b7d-69f3-4806-b6ee-bb12e14b0d82 None None] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:51:51.207964 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc557b7d-69f3-4806-b6ee-bb12e14b0d82 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:51:51.208451 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc557b7d-69f3-4806-b6ee-bb12e14b0d82 None None] PATH_INFO: `/` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:51:51.209755 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:51:51.210559 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 194/401] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 26 10:51:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 10:51:51.216997 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1c92cc-5073-4917-ba09-0b40a9d0d4d7 None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:51:51.219386 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1c92cc-5073-4917-ba09-0b40a9d0d4d7 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:51:51.219655 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1c92cc-5073-4917-ba09-0b40a9d0d4d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:51:51.248257 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-0b1c92cc-5073-4917-ba09-0b40a9d0d4d7 None None] MFA Rules not processed for user `ca155a48e2024c32af3fd2e19e5fac11`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 10:51:51.249420 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-0b1c92cc-5073-4917-ba09-0b40a9d0d4d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 10:51:51.265542 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0c3fae-4b67-4e2b-8556-16a45f0af291 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:51:51.304743 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:51:51.307650 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da059d96-2b2a-4e2b-8140-89fe27a127aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:51:51.308285 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da059d96-2b2a-4e2b-8140-89fe27a127aa None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:51:51.308560 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da059d96-2b2a-4e2b-8140-89fe27a127aa None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:51:51.308894 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da059d96-2b2a-4e2b-8140-89fe27a127aa None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:51:51.320962 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da059d96-2b2a-4e2b-8140-89fe27a127aa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 10:51:51.325828 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application [None req-da059d96-2b2a-4e2b-8140-89fe27a127aa None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 26 10:51:51.325828 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 10:51:51.325828 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 10:51:51.325828 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 10:51:51.325828 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 10:51:51.325828 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 10:51:51.325828 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 10:51:51.325828 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 10:51:51.325828 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 10:51:51.325828 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 10:51:51.325828 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 10:51:51.325828 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 10:51:51.325828 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 10:51:51.325828 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 10:51:51.325828 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 10:51:51.325828 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 10:51:51.326858 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 10:51:51.326858 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 10:51:51.326858 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 10:51:51.326858 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 10:51:51.326858 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 10:51:51.326858 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 10:51:51.326858 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 10:51:51.326858 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 10:51:51.326858 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 10:51:51.326858 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 10:51:51.326858 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 10:51:51.326858 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application with Lock( Aug 26 10:51:51.326858 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 10:51:51.326858 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._enter() Aug 26 10:51:51.326858 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 10:51:51.326858 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 10:51:51.327652 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 10:51:51.327652 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.creator() Aug 26 10:51:51.327652 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 10:51:51.327652 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application created_value = creator( Aug 26 10:51:51.327652 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 10:51:51.327652 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 10:51:51.327652 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 10:51:51.327652 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 10:51:51.327652 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 10:51:51.327652 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 10:51:51.327652 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 26 10:51:51.327652 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Aug 26 10:51:51.327652 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 208/402] 10.222.0.40 () {56 vars in 1114 bytes} [Fri Aug 26 10:51:51 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 10:51:51.328300 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da059d96-2b2a-4e2b-8140-89fe27a127aa None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:51:51.343172 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 195/403] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 26 10:51:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 10:51:51.344272 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b1c92cc-5073-4917-ba09-0b40a9d0d4d7 None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:51:51.510779 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:51:51.519962 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da059d96-2b2a-4e2b-8140-89fe27a127aa None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:51:51.559717 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b1c92cc-5073-4917-ba09-0b40a9d0d4d7 None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:51:51.592152 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb4e8c3-15a1-4072-8b69-d25bb4bb8b8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:51:51.594247 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb4e8c3-15a1-4072-8b69-d25bb4bb8b8c None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:51:51.594914 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb4e8c3-15a1-4072-8b69-d25bb4bb8b8c None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:51:51.596989 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb4e8c3-15a1-4072-8b69-d25bb4bb8b8c None admin] PATH_INFO: `/v3/groups/9b19097b64bb472a99936052ecdf4232` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:51:51.598553 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb4e8c3-15a1-4072-8b69-d25bb4bb8b8c None admin] RBAC: Authorizing `identity:get_group(group_id=9b19097b64bb472a99936052ecdf4232)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:51:51.602752 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb4e8c3-15a1-4072-8b69-d25bb4bb8b8c None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:51:51.604559 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 209/404] 10.222.0.40 () {56 vars in 1180 bytes} [Fri Aug 26 10:51:51 2022] GET /identity/v3/groups/9b19097b64bb472a99936052ecdf4232 => generated 226 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 10:51:51.614535 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb4e8c3-15a1-4072-8b69-d25bb4bb8b8c None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:51:51.636134 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a9524f-1e8d-4b0a-a539-e8ebcb49c57f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:51:51.636762 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a9524f-1e8d-4b0a-a539-e8ebcb49c57f None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:51:51.637042 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a9524f-1e8d-4b0a-a539-e8ebcb49c57f None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:51:51.637313 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a9524f-1e8d-4b0a-a539-e8ebcb49c57f None admin] PATH_INFO: `/v3/roles` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:51:51.640771 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a9524f-1e8d-4b0a-a539-e8ebcb49c57f None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 10:51:51.641080 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a9524f-1e8d-4b0a-a539-e8ebcb49c57f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:51:51.644646 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a9524f-1e8d-4b0a-a539-e8ebcb49c57f None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:51:51.665421 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 196/405] 10.222.0.40 () {56 vars in 1111 bytes} [Fri Aug 26 10:51:51 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 321 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 10:51:51.666374 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a9524f-1e8d-4b0a-a539-e8ebcb49c57f None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:51:51.725745 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:51:51.742461 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb4e8c3-15a1-4072-8b69-d25bb4bb8b8c None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:51:51.778230 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ca01ab-82b3-4ca8-a81f-008d588e1883 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:51:51.779153 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ca01ab-82b3-4ca8-a81f-008d588e1883 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:51:51.779606 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ca01ab-82b3-4ca8-a81f-008d588e1883 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:51:51.780052 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ca01ab-82b3-4ca8-a81f-008d588e1883 None admin] PATH_INFO: `/v3/projects/da070dea52de4d71952ed50827d44a1c` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:51:51.781372 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ca01ab-82b3-4ca8-a81f-008d588e1883 None admin] RBAC: Authorizing `identity:get_project(project_id=da070dea52de4d71952ed50827d44a1c)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:51:51.783727 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ca01ab-82b3-4ca8-a81f-008d588e1883 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:51:51.786099 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 210/406] 10.222.0.40 () {56 vars in 1186 bytes} [Fri Aug 26 10:51:51 2022] GET /identity/v3/projects/da070dea52de4d71952ed50827d44a1c => generated 298 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 10:51:51.787045 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ca01ab-82b3-4ca8-a81f-008d588e1883 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:51:51.797557 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a9524f-1e8d-4b0a-a539-e8ebcb49c57f None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:51:51.820944 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:51:51.845078 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aed943d-b161-4fea-a40a-dbd5957b842c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:51:51.845825 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aed943d-b161-4fea-a40a-dbd5957b842c None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:51:51.846242 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aed943d-b161-4fea-a40a-dbd5957b842c None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:51:51.846573 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aed943d-b161-4fea-a40a-dbd5957b842c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:51:51.861744 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aed943d-b161-4fea-a40a-dbd5957b842c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 10:51:51.869611 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application [None req-3aed943d-b161-4fea-a40a-dbd5957b842c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 26 10:51:51.869611 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 10:51:51.869611 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 10:51:51.869611 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 10:51:51.869611 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 10:51:51.869611 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 10:51:51.869611 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 10:51:51.869611 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 10:51:51.869611 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 10:51:51.869611 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 10:51:51.869611 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 10:51:51.869611 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 10:51:51.869611 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 10:51:51.869611 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 10:51:51.869611 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 10:51:51.869611 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 10:51:51.870548 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 10:51:51.870548 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 10:51:51.870548 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 10:51:51.870548 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 10:51:51.870548 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 10:51:51.870548 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 10:51:51.870548 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 10:51:51.870548 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 10:51:51.870548 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 10:51:51.870548 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 10:51:51.870548 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 10:51:51.870548 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application with Lock( Aug 26 10:51:51.870548 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 10:51:51.870548 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._enter() Aug 26 10:51:51.870548 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 10:51:51.870548 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 10:51:51.871314 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 10:51:51.871314 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.creator() Aug 26 10:51:51.871314 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 10:51:51.871314 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application created_value = creator( Aug 26 10:51:51.871314 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 10:51:51.871314 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 10:51:51.871314 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 10:51:51.871314 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 10:51:51.871314 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 10:51:51.871314 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 10:51:51.871314 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 26 10:51:51.871314 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Aug 26 10:51:51.871314 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 197/407] 10.222.0.40 () {56 vars in 1096 bytes} [Fri Aug 26 10:51:51 2022] GET /identity/v3/roles/admin => generated 83 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 10:51:51.872052 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aed943d-b161-4fea-a40a-dbd5957b842c None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:51:51.908785 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ca01ab-82b3-4ca8-a81f-008d588e1883 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:51:51.944872 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64b75ad-84fc-4e98-93af-dc2d271d1fe4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:51:51.945596 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64b75ad-84fc-4e98-93af-dc2d271d1fe4 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:51:51.945942 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64b75ad-84fc-4e98-93af-dc2d271d1fe4 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:51:51.946306 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64b75ad-84fc-4e98-93af-dc2d271d1fe4 None admin] PATH_INFO: `/v3/groups/9b19097b64bb472a99936052ecdf4232` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:51:51.947427 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d64b75ad-84fc-4e98-93af-dc2d271d1fe4 None admin] RBAC: Authorizing `identity:get_group(group_id=9b19097b64bb472a99936052ecdf4232)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:51:51.948853 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d64b75ad-84fc-4e98-93af-dc2d271d1fe4 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:51:51.950749 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 211/408] 10.222.0.40 () {56 vars in 1180 bytes} [Fri Aug 26 10:51:51 2022] GET /identity/v3/groups/9b19097b64bb472a99936052ecdf4232 => generated 226 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 10:51:51.951589 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64b75ad-84fc-4e98-93af-dc2d271d1fe4 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:51:52.001172 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aed943d-b161-4fea-a40a-dbd5957b842c None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:51:52.004038 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:51:52.041179 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40e605c-bb34-4fd6-9d4c-da616e3647fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:51:52.041762 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40e605c-bb34-4fd6-9d4c-da616e3647fd None admin] REQUEST_METHOD: `PUT` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:51:52.042098 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40e605c-bb34-4fd6-9d4c-da616e3647fd None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:51:52.042345 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40e605c-bb34-4fd6-9d4c-da616e3647fd None admin] PATH_INFO: `/v3/projects/da070dea52de4d71952ed50827d44a1c/groups/9b19097b64bb472a99936052ecdf4232/roles/9c439f898126444f92abb01910f1d02b` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:51:52.044837 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f40e605c-bb34-4fd6-9d4c-da616e3647fd None admin] RBAC: Authorizing `identity:create_grant(project_id=da070dea52de4d71952ed50827d44a1c, group_id=9b19097b64bb472a99936052ecdf4232, role_id=9c439f898126444f92abb01910f1d02b)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:51:52.046304 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f40e605c-bb34-4fd6-9d4c-da616e3647fd None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:51:52.060043 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 198/409] 10.222.0.40 () {58 vars in 1442 bytes} [Fri Aug 26 10:51:51 2022] PUT /identity/v3/projects/da070dea52de4d71952ed50827d44a1c/groups/9b19097b64bb472a99936052ecdf4232/roles/9c439f898126444f92abb01910f1d02b => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 10:51:52.060840 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40e605c-bb34-4fd6-9d4c-da616e3647fd None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:51:52.071682 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:51:52.077335 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64b75ad-84fc-4e98-93af-dc2d271d1fe4 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:51:52.140265 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2d1477-748a-4ba3-ad71-a30563fa944b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:51:52.142153 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2d1477-748a-4ba3-ad71-a30563fa944b None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:51:52.142748 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2d1477-748a-4ba3-ad71-a30563fa944b None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:51:52.143296 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2d1477-748a-4ba3-ad71-a30563fa944b None admin] PATH_INFO: `/v3/roles` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:51:52.144886 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc2d1477-748a-4ba3-ad71-a30563fa944b None admin] RBAC: Adding query filter params (name=admin) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 10:51:52.145568 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc2d1477-748a-4ba3-ad71-a30563fa944b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:51:52.147671 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc2d1477-748a-4ba3-ad71-a30563fa944b None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:51:52.160563 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 212/410] 10.222.0.40 () {56 vars in 1099 bytes} [Fri Aug 26 10:51:51 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 10:51:52.162523 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cec7a10-8a58-4ed1-9ac0-13a8b3fb726a None None] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:51:52.163268 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cec7a10-8a58-4ed1-9ac0-13a8b3fb726a None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:51:52.163721 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cec7a10-8a58-4ed1-9ac0-13a8b3fb726a None None] PATH_INFO: `/` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:51:52.166334 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 213/411] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 26 10:51:52 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 10:51:52.167353 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cec7a10-8a58-4ed1-9ac0-13a8b3fb726a None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:51:52.192854 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40e605c-bb34-4fd6-9d4c-da616e3647fd None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:51:52.239320 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f66de8d-7d5d-4a68-9529-e3839f3a0287 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:51:52.239983 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f66de8d-7d5d-4a68-9529-e3839f3a0287 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:51:52.240362 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f66de8d-7d5d-4a68-9529-e3839f3a0287 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:51:52.240724 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f66de8d-7d5d-4a68-9529-e3839f3a0287 None admin] PATH_INFO: `/v3/projects/15667f91e7f6473cb9aab41830d80f5e` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:51:52.242054 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f66de8d-7d5d-4a68-9529-e3839f3a0287 None admin] RBAC: Authorizing `identity:get_project(project_id=15667f91e7f6473cb9aab41830d80f5e)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:51:52.243518 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f66de8d-7d5d-4a68-9529-e3839f3a0287 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:51:52.245561 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 199/412] 10.222.0.40 () {56 vars in 1186 bytes} [Fri Aug 26 10:51:52 2022] GET /identity/v3/projects/15667f91e7f6473cb9aab41830d80f5e => generated 302 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 10:51:52.246702 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5be8943-b97a-4f4d-96f3-a0ed27b50064 None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:51:52.250432 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5be8943-b97a-4f4d-96f3-a0ed27b50064 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:51:52.250994 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5be8943-b97a-4f4d-96f3-a0ed27b50064 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:51:52.273705 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-b5be8943-b97a-4f4d-96f3-a0ed27b50064 None None] MFA Rules not processed for user `ca155a48e2024c32af3fd2e19e5fac11`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 10:51:52.275505 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-b5be8943-b97a-4f4d-96f3-a0ed27b50064 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 10:51:52.303261 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cec7a10-8a58-4ed1-9ac0-13a8b3fb726a None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:51:52.309342 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:51:52.352284 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f7111b-129b-4743-bd62-bedbcf378ca1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:51:52.353947 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f7111b-129b-4743-bd62-bedbcf378ca1 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:51:52.354659 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f7111b-129b-4743-bd62-bedbcf378ca1 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:51:52.355086 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f7111b-129b-4743-bd62-bedbcf378ca1 None admin] PATH_INFO: `/v3/groups/907d2ba571f44f6c8cf2a75be1eda1e0` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:51:52.356602 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f7111b-129b-4743-bd62-bedbcf378ca1 None admin] RBAC: Authorizing `identity:get_group(group_id=907d2ba571f44f6c8cf2a75be1eda1e0)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:51:52.359175 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f7111b-129b-4743-bd62-bedbcf378ca1 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:51:52.362086 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 214/413] 10.222.0.40 () {56 vars in 1180 bytes} [Fri Aug 26 10:51:52 2022] GET /identity/v3/groups/907d2ba571f44f6c8cf2a75be1eda1e0 => generated 229 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 10:51:52.363091 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f7111b-129b-4743-bd62-bedbcf378ca1 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:51:52.374066 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 200/414] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 26 10:51:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 10:51:52.375133 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5be8943-b97a-4f4d-96f3-a0ed27b50064 None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:51:52.434333 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:51:52.530812 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5be8943-b97a-4f4d-96f3-a0ed27b50064 None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:51:52.569107 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f7111b-129b-4743-bd62-bedbcf378ca1 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:51:52.569345 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369f3c72-9897-40e0-8988-b436e7282131 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:51:52.570006 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369f3c72-9897-40e0-8988-b436e7282131 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:51:52.570319 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369f3c72-9897-40e0-8988-b436e7282131 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:51:52.570596 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369f3c72-9897-40e0-8988-b436e7282131 None admin] PATH_INFO: `/v3/projects/f3942b05c97e4a1ca048ce3a0379b7a7` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:51:52.576248 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-369f3c72-9897-40e0-8988-b436e7282131 None admin] RBAC: Authorizing `identity:get_project(project_id=f3942b05c97e4a1ca048ce3a0379b7a7)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:51:52.576248 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-369f3c72-9897-40e0-8988-b436e7282131 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:51:52.576248 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 201/415] 10.222.0.40 () {56 vars in 1186 bytes} [Fri Aug 26 10:51:52 2022] GET /identity/v3/projects/f3942b05c97e4a1ca048ce3a0379b7a7 => generated 344 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 10:51:52.581343 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369f3c72-9897-40e0-8988-b436e7282131 None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:51:52.613474 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:51:52.625257 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c74963b-df46-4e48-a25d-b6086960aedb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:51:52.625927 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c74963b-df46-4e48-a25d-b6086960aedb None admin] REQUEST_METHOD: `PUT` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:51:52.626238 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c74963b-df46-4e48-a25d-b6086960aedb None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:51:52.626508 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c74963b-df46-4e48-a25d-b6086960aedb None admin] PATH_INFO: `/v3/projects/15667f91e7f6473cb9aab41830d80f5e/groups/9b19097b64bb472a99936052ecdf4232/roles/9c439f898126444f92abb01910f1d02b` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:51:52.630878 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c74963b-df46-4e48-a25d-b6086960aedb None admin] RBAC: Authorizing `identity:create_grant(project_id=15667f91e7f6473cb9aab41830d80f5e, group_id=9b19097b64bb472a99936052ecdf4232, role_id=9c439f898126444f92abb01910f1d02b)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:51:52.632823 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c74963b-df46-4e48-a25d-b6086960aedb None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:51:52.648919 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 215/416] 10.222.0.40 () {58 vars in 1442 bytes} [Fri Aug 26 10:51:52 2022] PUT /identity/v3/projects/15667f91e7f6473cb9aab41830d80f5e/groups/9b19097b64bb472a99936052ecdf4232/roles/9c439f898126444f92abb01910f1d02b => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 10:51:52.695199 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369f3c72-9897-40e0-8988-b436e7282131 None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:51:52.734191 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784d61c0-a174-443a-bca3-6cbdfaf5cde1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:51:52.735250 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784d61c0-a174-443a-bca3-6cbdfaf5cde1 None admin] REQUEST_METHOD: `PUT` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:51:52.735735 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784d61c0-a174-443a-bca3-6cbdfaf5cde1 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:51:52.736829 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784d61c0-a174-443a-bca3-6cbdfaf5cde1 None admin] PATH_INFO: `/v3/projects/f3942b05c97e4a1ca048ce3a0379b7a7/groups/907d2ba571f44f6c8cf2a75be1eda1e0/roles/1f97e4602e2e4d6498913a81f9ae8d6a` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:51:52.739278 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-784d61c0-a174-443a-bca3-6cbdfaf5cde1 None admin] RBAC: Authorizing `identity:create_grant(project_id=f3942b05c97e4a1ca048ce3a0379b7a7, group_id=907d2ba571f44f6c8cf2a75be1eda1e0, role_id=1f97e4602e2e4d6498913a81f9ae8d6a)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:51:52.740967 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-784d61c0-a174-443a-bca3-6cbdfaf5cde1 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:51:52.752266 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c74963b-df46-4e48-a25d-b6086960aedb None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:51:52.764520 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 202/417] 10.222.0.40 () {58 vars in 1442 bytes} [Fri Aug 26 10:51:52 2022] PUT /identity/v3/projects/f3942b05c97e4a1ca048ce3a0379b7a7/groups/907d2ba571f44f6c8cf2a75be1eda1e0/roles/1f97e4602e2e4d6498913a81f9ae8d6a => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 26 10:51:52.869231 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:51:52.880676 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c74963b-df46-4e48-a25d-b6086960aedb None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:51:52.927872 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b581d5f-7ee9-4b75-b252-68f55dca2e75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:51:52.928677 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b581d5f-7ee9-4b75-b252-68f55dca2e75 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:51:52.928960 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b581d5f-7ee9-4b75-b252-68f55dca2e75 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:51:52.929227 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b581d5f-7ee9-4b75-b252-68f55dca2e75 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:51:52.941867 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b581d5f-7ee9-4b75-b252-68f55dca2e75 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 10:51:52.947239 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application [None req-0b581d5f-7ee9-4b75-b252-68f55dca2e75 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 26 10:51:52.947239 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 10:51:52.947239 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 10:51:52.947239 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 10:51:52.947239 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 10:51:52.947239 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 10:51:52.947239 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 10:51:52.947239 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 10:51:52.947239 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 10:51:52.947239 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 10:51:52.947239 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 10:51:52.947239 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 10:51:52.947239 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 10:51:52.947239 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 10:51:52.947239 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 10:51:52.947239 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 10:51:52.948185 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 10:51:52.948185 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 10:51:52.948185 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 10:51:52.948185 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 10:51:52.948185 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 10:51:52.948185 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 10:51:52.948185 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 10:51:52.948185 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 10:51:52.948185 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 10:51:52.948185 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 10:51:52.948185 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 10:51:52.948185 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application with Lock( Aug 26 10:51:52.948185 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 10:51:52.948185 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._enter() Aug 26 10:51:52.948185 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 10:51:52.948185 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 10:51:52.949027 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 10:51:52.949027 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.creator() Aug 26 10:51:52.949027 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 10:51:52.949027 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application created_value = creator( Aug 26 10:51:52.949027 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 10:51:52.949027 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 10:51:52.949027 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 10:51:52.949027 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 10:51:52.949027 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 10:51:52.949027 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 10:51:52.949027 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 26 10:51:52.949027 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Aug 26 10:51:52.949027 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 216/418] 10.222.0.40 () {56 vars in 1099 bytes} [Fri Aug 26 10:51:52 2022] GET /identity/v3/roles/member => generated 84 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 10:51:52.955295 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784d61c0-a174-443a-bca3-6cbdfaf5cde1 None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:51:53.056429 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:51:53.076325 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784d61c0-a174-443a-bca3-6cbdfaf5cde1 None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:51:53.134720 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7be33d5-cf27-4b74-8680-94d120011b30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:51:53.135341 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7be33d5-cf27-4b74-8680-94d120011b30 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:51:53.135617 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7be33d5-cf27-4b74-8680-94d120011b30 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:51:53.135883 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7be33d5-cf27-4b74-8680-94d120011b30 None admin] PATH_INFO: `/v3/roles` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:51:53.136550 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7be33d5-cf27-4b74-8680-94d120011b30 None admin] RBAC: Adding query filter params (name=member) {{(pid=81423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 10:51:53.136842 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7be33d5-cf27-4b74-8680-94d120011b30 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:51:53.138246 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7be33d5-cf27-4b74-8680-94d120011b30 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:51:53.148055 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 203/419] 10.222.0.40 () {56 vars in 1101 bytes} [Fri Aug 26 10:51:52 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 10:51:53.153588 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b581d5f-7ee9-4b75-b252-68f55dca2e75 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:51:53.224736 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e3059b-eeff-4581-bdd3-9723f6805389 None None] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:51:53.225049 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e3059b-eeff-4581-bdd3-9723f6805389 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:51:53.225284 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e3059b-eeff-4581-bdd3-9723f6805389 None None] PATH_INFO: `/` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:51:53.226974 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 204/420] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 26 10:51:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 10:51:53.232311 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19b5ee9-84cb-4d9b-a88f-93d2b12a6489 None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:51:53.232587 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19b5ee9-84cb-4d9b-a88f-93d2b12a6489 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:51:53.233116 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19b5ee9-84cb-4d9b-a88f-93d2b12a6489 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:51:53.251313 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-d19b5ee9-84cb-4d9b-a88f-93d2b12a6489 None None] MFA Rules not processed for user `ca155a48e2024c32af3fd2e19e5fac11`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 10:51:53.252532 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-d19b5ee9-84cb-4d9b-a88f-93d2b12a6489 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 10:51:53.295084 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b581d5f-7ee9-4b75-b252-68f55dca2e75 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:51:53.362796 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7990d06f-68b6-4bb8-95e2-0945c9873367 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:51:53.363449 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7990d06f-68b6-4bb8-95e2-0945c9873367 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:51:53.363702 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7990d06f-68b6-4bb8-95e2-0945c9873367 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:51:53.363936 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7990d06f-68b6-4bb8-95e2-0945c9873367 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:51:53.368554 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 205/421] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 26 10:51:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 26 10:51:53.406421 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:51:53.476003 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7990d06f-68b6-4bb8-95e2-0945c9873367 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:51:53.477478 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7990d06f-68b6-4bb8-95e2-0945c9873367 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:51:53.588285 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 217/422] 10.222.0.40 () {58 vars in 1282 bytes} [Fri Aug 26 10:51:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 435 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 26 10:51:53.593709 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19b5ee9-84cb-4d9b-a88f-93d2b12a6489 None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:51:53.655074 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7990d06f-68b6-4bb8-95e2-0945c9873367 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:51:53.704202 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:51:53.732747 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19b5ee9-84cb-4d9b-a88f-93d2b12a6489 None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:51:53.765846 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:51:53.772405 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f90bdf-45d4-4344-8862-75278305a933 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:51:53.773048 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f90bdf-45d4-4344-8862-75278305a933 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:51:53.773323 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f90bdf-45d4-4344-8862-75278305a933 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:51:53.773594 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f90bdf-45d4-4344-8862-75278305a933 None admin] PATH_INFO: `/v3/projects/da070dea52de4d71952ed50827d44a1c` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:51:53.774759 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f90bdf-45d4-4344-8862-75278305a933 None admin] RBAC: Authorizing `identity:get_project(project_id=da070dea52de4d71952ed50827d44a1c)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:51:53.776183 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f90bdf-45d4-4344-8862-75278305a933 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:51:53.778178 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 206/423] 10.222.0.40 () {56 vars in 1186 bytes} [Fri Aug 26 10:51:53 2022] GET /identity/v3/projects/da070dea52de4d71952ed50827d44a1c => generated 298 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 10:51:53.790887 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f90bdf-45d4-4344-8862-75278305a933 None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:51:53.859364 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7990d06f-68b6-4bb8-95e2-0945c9873367 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:51:53.895100 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7dd59d-04b0-4f57-8273-e1e821cc780e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:51:53.895727 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7dd59d-04b0-4f57-8273-e1e821cc780e None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:51:53.896006 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7dd59d-04b0-4f57-8273-e1e821cc780e None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:51:53.896322 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7dd59d-04b0-4f57-8273-e1e821cc780e None admin] PATH_INFO: `/v3/roles/member` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:51:53.906619 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc7dd59d-04b0-4f57-8273-e1e821cc780e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 10:51:53.912497 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application [None req-bc7dd59d-04b0-4f57-8273-e1e821cc780e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 26 10:51:53.912497 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 10:51:53.912497 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 10:51:53.912497 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 10:51:53.912497 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 10:51:53.912497 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 10:51:53.912497 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 10:51:53.912497 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 10:51:53.912497 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 10:51:53.912497 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 10:51:53.912497 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 10:51:53.912497 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 10:51:53.912497 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 10:51:53.912497 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 10:51:53.912497 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 10:51:53.912497 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 10:51:53.913599 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 10:51:53.913599 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 10:51:53.913599 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 10:51:53.913599 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 10:51:53.913599 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 10:51:53.913599 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 10:51:53.913599 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 10:51:53.913599 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 10:51:53.913599 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 10:51:53.913599 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 10:51:53.913599 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 10:51:53.913599 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application with Lock( Aug 26 10:51:53.913599 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 10:51:53.913599 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._enter() Aug 26 10:51:53.913599 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 10:51:53.913599 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 10:51:53.914617 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 10:51:53.914617 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.creator() Aug 26 10:51:53.914617 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 10:51:53.914617 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application created_value = creator( Aug 26 10:51:53.914617 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 10:51:53.914617 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 10:51:53.914617 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 10:51:53.914617 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 10:51:53.914617 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 10:51:53.914617 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 10:51:53.914617 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 26 10:51:53.914617 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Aug 26 10:51:53.914617 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 218/424] 10.222.0.40 () {56 vars in 1099 bytes} [Fri Aug 26 10:51:53 2022] GET /identity/v3/roles/member => generated 84 bytes in 259 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 26 10:51:53.919208 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7dd59d-04b0-4f57-8273-e1e821cc780e None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:51:54.034338 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7dd59d-04b0-4f57-8273-e1e821cc780e None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:51:54.052689 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:51:54.079190 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9e8133-755f-4533-a64d-187b08f8ef9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:51:54.081591 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9e8133-755f-4533-a64d-187b08f8ef9b None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:51:54.081894 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9e8133-755f-4533-a64d-187b08f8ef9b None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:51:54.082201 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9e8133-755f-4533-a64d-187b08f8ef9b None admin] PATH_INFO: `/v3/roles` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:51:54.082881 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd9e8133-755f-4533-a64d-187b08f8ef9b None admin] RBAC: Adding query filter params (name=member) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 10:51:54.083185 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd9e8133-755f-4533-a64d-187b08f8ef9b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:51:54.084572 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd9e8133-755f-4533-a64d-187b08f8ef9b None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:51:54.098444 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f90bdf-45d4-4344-8862-75278305a933 None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:51:54.109504 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 219/425] 10.222.0.40 () {56 vars in 1101 bytes} [Fri Aug 26 10:51:53 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 10:51:54.114824 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9e8133-755f-4533-a64d-187b08f8ef9b None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:51:54.164331 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874d8ce5-e370-42bb-9ec3-762da98f5d05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:51:54.164970 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874d8ce5-e370-42bb-9ec3-762da98f5d05 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:51:54.165246 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874d8ce5-e370-42bb-9ec3-762da98f5d05 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:51:54.165516 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874d8ce5-e370-42bb-9ec3-762da98f5d05 None admin] PATH_INFO: `/v3/groups/9b19097b64bb472a99936052ecdf4232` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:51:54.167372 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-874d8ce5-e370-42bb-9ec3-762da98f5d05 None admin] RBAC: Authorizing `identity:get_group(group_id=9b19097b64bb472a99936052ecdf4232)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:51:54.168810 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-874d8ce5-e370-42bb-9ec3-762da98f5d05 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:51:54.172362 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 207/426] 10.222.0.40 () {56 vars in 1180 bytes} [Fri Aug 26 10:51:53 2022] GET /identity/v3/groups/9b19097b64bb472a99936052ecdf4232 => generated 226 bytes in 380 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 10:51:54.178292 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874d8ce5-e370-42bb-9ec3-762da98f5d05 None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:51:54.218385 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:51:54.253558 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9e8133-755f-4533-a64d-187b08f8ef9b None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:51:54.289893 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179eef13-cbfc-4977-851e-e1f53d387474 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:51:54.290492 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179eef13-cbfc-4977-851e-e1f53d387474 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:51:54.290771 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179eef13-cbfc-4977-851e-e1f53d387474 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:51:54.291039 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179eef13-cbfc-4977-851e-e1f53d387474 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:51:54.298307 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874d8ce5-e370-42bb-9ec3-762da98f5d05 None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:51:54.344971 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:51:54.350662 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f41cc50-c6dd-4896-b63a-96ea7539c156 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:51:54.351649 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f41cc50-c6dd-4896-b63a-96ea7539c156 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:51:54.351920 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f41cc50-c6dd-4896-b63a-96ea7539c156 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:51:54.352500 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f41cc50-c6dd-4896-b63a-96ea7539c156 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:51:54.353290 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f41cc50-c6dd-4896-b63a-96ea7539c156 None admin] RBAC: Adding query filter params (group.id=9b19097b64bb472a99936052ecdf4232, role.id=23e174ce743b46eabe618c212691da55, scope.project.id=da070dea52de4d71952ed50827d44a1c) {{(pid=81423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 10:51:54.353756 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f41cc50-c6dd-4896-b63a-96ea7539c156 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:51:54.355619 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f41cc50-c6dd-4896-b63a-96ea7539c156 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:51:54.365348 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 208/427] 10.222.0.40 () {56 vars in 1376 bytes} [Fri Aug 26 10:51:54 2022] GET /identity/v3/role_assignments?group.id=9b19097b64bb472a99936052ecdf4232&scope.project.id=da070dea52de4d71952ed50827d44a1c&role.id=23e174ce743b46eabe618c212691da55 => generated 611 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 10:51:54.371079 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d761ec-4199-459b-9b58-f992e0300c29 None None] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:51:54.371431 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d761ec-4199-459b-9b58-f992e0300c29 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:51:54.371586 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d761ec-4199-459b-9b58-f992e0300c29 None None] PATH_INFO: `/` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:51:54.378477 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 209/428] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 26 10:51:54 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 10:51:54.381416 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba333e5b-4f61-406f-9ca6-2040aa3f2c41 None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:51:54.381691 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba333e5b-4f61-406f-9ca6-2040aa3f2c41 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:51:54.381967 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba333e5b-4f61-406f-9ca6-2040aa3f2c41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:51:54.396896 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-179eef13-cbfc-4977-851e-e1f53d387474 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:51:54.398544 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-179eef13-cbfc-4977-851e-e1f53d387474 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:51:54.420838 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-ba333e5b-4f61-406f-9ca6-2040aa3f2c41 None None] MFA Rules not processed for user `ca155a48e2024c32af3fd2e19e5fac11`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 10:51:54.422086 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-ba333e5b-4f61-406f-9ca6-2040aa3f2c41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 10:51:54.562663 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 210/429] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 26 10:51:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 10:51:54.576361 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 220/430] 10.222.0.40 () {58 vars in 1282 bytes} [Fri Aug 26 10:51:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 462 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 26 10:51:54.583235 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba333e5b-4f61-406f-9ca6-2040aa3f2c41 None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:51:54.601102 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:51:54.693511 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba333e5b-4f61-406f-9ca6-2040aa3f2c41 None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:51:54.731745 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd63a66-8bda-4614-b1bd-fabb54802394 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:51:54.732389 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd63a66-8bda-4614-b1bd-fabb54802394 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:51:54.732670 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd63a66-8bda-4614-b1bd-fabb54802394 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:51:54.733302 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd63a66-8bda-4614-b1bd-fabb54802394 None admin] PATH_INFO: `/v3/projects/15667f91e7f6473cb9aab41830d80f5e` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:51:54.734467 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebd63a66-8bda-4614-b1bd-fabb54802394 None admin] RBAC: Authorizing `identity:get_project(project_id=15667f91e7f6473cb9aab41830d80f5e)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:51:54.735896 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebd63a66-8bda-4614-b1bd-fabb54802394 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:51:54.742166 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 211/431] 10.222.0.40 () {56 vars in 1186 bytes} [Fri Aug 26 10:51:54 2022] GET /identity/v3/projects/15667f91e7f6473cb9aab41830d80f5e => generated 302 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 10:51:54.744486 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179eef13-cbfc-4977-851e-e1f53d387474 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:51:54.836237 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:51:54.869962 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd63a66-8bda-4614-b1bd-fabb54802394 None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:51:54.873673 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179eef13-cbfc-4977-851e-e1f53d387474 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:51:54.909040 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ad4436-987d-44b0-9a4e-74e8842474ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:51:54.909669 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ad4436-987d-44b0-9a4e-74e8842474ed None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:51:54.909944 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ad4436-987d-44b0-9a4e-74e8842474ed None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:51:54.910240 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ad4436-987d-44b0-9a4e-74e8842474ed None admin] PATH_INFO: `/v3/groups/9b19097b64bb472a99936052ecdf4232` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:51:54.911386 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ad4436-987d-44b0-9a4e-74e8842474ed None admin] RBAC: Authorizing `identity:get_group(group_id=9b19097b64bb472a99936052ecdf4232)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:51:54.912792 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ad4436-987d-44b0-9a4e-74e8842474ed None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:51:54.914564 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 221/432] 10.222.0.40 () {56 vars in 1180 bytes} [Fri Aug 26 10:51:54 2022] GET /identity/v3/groups/9b19097b64bb472a99936052ecdf4232 => generated 226 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 10:51:54.920161 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ad4436-987d-44b0-9a4e-74e8842474ed None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:51:54.989661 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:51:55.014122 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd63a66-8bda-4614-b1bd-fabb54802394 None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:51:55.024451 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ad4436-987d-44b0-9a4e-74e8842474ed None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:51:55.062563 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b97c1ae-19b7-4e3e-a2f4-e796123da997 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:51:55.063122 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426ca275-9030-4a28-a14d-aaa375ee5a0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:51:55.063731 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426ca275-9030-4a28-a14d-aaa375ee5a0c None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:51:55.064008 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426ca275-9030-4a28-a14d-aaa375ee5a0c None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:51:55.064328 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426ca275-9030-4a28-a14d-aaa375ee5a0c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:51:55.064658 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b97c1ae-19b7-4e3e-a2f4-e796123da997 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:51:55.065049 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-426ca275-9030-4a28-a14d-aaa375ee5a0c None admin] RBAC: Adding query filter params (group.id=9b19097b64bb472a99936052ecdf4232, role.id=23e174ce743b46eabe618c212691da55, scope.project.id=15667f91e7f6473cb9aab41830d80f5e) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 10:51:55.065351 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-426ca275-9030-4a28-a14d-aaa375ee5a0c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:51:55.065650 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b97c1ae-19b7-4e3e-a2f4-e796123da997 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:51:55.066136 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b97c1ae-19b7-4e3e-a2f4-e796123da997 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:51:55.066805 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-426ca275-9030-4a28-a14d-aaa375ee5a0c None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:51:55.076736 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 222/433] 10.222.0.40 () {56 vars in 1376 bytes} [Fri Aug 26 10:51:54 2022] GET /identity/v3/role_assignments?group.id=9b19097b64bb472a99936052ecdf4232&scope.project.id=15667f91e7f6473cb9aab41830d80f5e&role.id=23e174ce743b46eabe618c212691da55 => generated 611 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 10:51:55.081120 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b97c1ae-19b7-4e3e-a2f4-e796123da997 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 10:51:55.087114 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application [None req-8b97c1ae-19b7-4e3e-a2f4-e796123da997 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 26 10:51:55.087114 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 10:51:55.087114 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 10:51:55.087114 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 10:51:55.087114 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 10:51:55.087114 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 10:51:55.087114 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 10:51:55.087114 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 10:51:55.087114 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 10:51:55.087114 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 10:51:55.087114 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 10:51:55.087114 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 10:51:55.087114 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 10:51:55.087114 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 10:51:55.087114 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 10:51:55.087114 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 10:51:55.088036 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 10:51:55.088036 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 10:51:55.088036 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 10:51:55.088036 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 10:51:55.088036 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 10:51:55.088036 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 10:51:55.088036 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 10:51:55.088036 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 10:51:55.088036 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 10:51:55.088036 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 10:51:55.088036 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 10:51:55.088036 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application with Lock( Aug 26 10:51:55.088036 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 10:51:55.088036 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._enter() Aug 26 10:51:55.088036 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 10:51:55.088036 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 10:51:55.088805 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 10:51:55.088805 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.creator() Aug 26 10:51:55.088805 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 10:51:55.088805 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application created_value = creator( Aug 26 10:51:55.088805 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 10:51:55.088805 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 10:51:55.088805 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 10:51:55.088805 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 10:51:55.088805 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 10:51:55.088805 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 10:51:55.088805 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 26 10:51:55.088805 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Aug 26 10:51:55.091717 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 212/434] 10.222.0.40 () {56 vars in 1114 bytes} [Fri Aug 26 10:51:54 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 10:51:55.098557 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426ca275-9030-4a28-a14d-aaa375ee5a0c None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:51:55.107495 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:51:55.231362 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69c6b48-b054-4160-904d-2c9dff2964d5 None None] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:51:55.231834 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69c6b48-b054-4160-904d-2c9dff2964d5 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:51:55.232246 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69c6b48-b054-4160-904d-2c9dff2964d5 None None] PATH_INFO: `/` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:51:55.239327 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 213/435] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 26 10:51:55 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 26 10:51:55.258020 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4aa192-a92c-4c05-9e1f-984aba053895 None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:51:55.258020 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4aa192-a92c-4c05-9e1f-984aba053895 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:51:55.258265 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4aa192-a92c-4c05-9e1f-984aba053895 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:51:55.268651 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426ca275-9030-4a28-a14d-aaa375ee5a0c None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:51:55.283937 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-eb4aa192-a92c-4c05-9e1f-984aba053895 None None] MFA Rules not processed for user `ca155a48e2024c32af3fd2e19e5fac11`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 10:51:55.285225 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-eb4aa192-a92c-4c05-9e1f-984aba053895 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 10:51:55.313193 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e968c7-4a02-4b8e-acb8-3170ca344f74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:51:55.313809 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e968c7-4a02-4b8e-acb8-3170ca344f74 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:51:55.314292 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e968c7-4a02-4b8e-acb8-3170ca344f74 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:51:55.314649 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e968c7-4a02-4b8e-acb8-3170ca344f74 None admin] PATH_INFO: `/v3/roles` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:51:55.315327 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e968c7-4a02-4b8e-acb8-3170ca344f74 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 10:51:55.315635 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e968c7-4a02-4b8e-acb8-3170ca344f74 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:51:55.317029 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e968c7-4a02-4b8e-acb8-3170ca344f74 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:51:55.326599 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 223/436] 10.222.0.40 () {56 vars in 1111 bytes} [Fri Aug 26 10:51:55 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 10:51:55.332979 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e968c7-4a02-4b8e-acb8-3170ca344f74 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:51:55.384103 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 214/437] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 26 10:51:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 10:51:55.385133 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c492e0-1512-459e-b10a-fa3ca429e496 None None] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:51:55.385411 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c492e0-1512-459e-b10a-fa3ca429e496 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:51:55.385676 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c492e0-1512-459e-b10a-fa3ca429e496 None None] PATH_INFO: `/` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:51:55.391668 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 215/438] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 26 10:51:55 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 10:51:55.397267 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4202b91-1bb8-475d-b5bb-e827ee473968 None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:51:55.397893 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4202b91-1bb8-475d-b5bb-e827ee473968 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:51:55.397893 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4202b91-1bb8-475d-b5bb-e827ee473968 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:51:55.416145 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-d4202b91-1bb8-475d-b5bb-e827ee473968 None None] MFA Rules not processed for user `ca155a48e2024c32af3fd2e19e5fac11`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 10:51:55.417338 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-d4202b91-1bb8-475d-b5bb-e827ee473968 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 10:51:55.419159 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:51:55.457841 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e968c7-4a02-4b8e-acb8-3170ca344f74 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:51:55.484068 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:51:55.490111 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 216/439] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 26 10:51:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 26 10:51:55.496533 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eecb1c4-f250-4055-8f04-9693500fcbdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:51:55.497145 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eecb1c4-f250-4055-8f04-9693500fcbdf None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:51:55.497424 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eecb1c4-f250-4055-8f04-9693500fcbdf None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:51:55.497696 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eecb1c4-f250-4055-8f04-9693500fcbdf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:51:55.563002 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eecb1c4-f250-4055-8f04-9693500fcbdf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:51:55.565006 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eecb1c4-f250-4055-8f04-9693500fcbdf None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:51:55.625967 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:51:55.662860 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 224/440] 10.222.0.40 () {58 vars in 1282 bytes} [Fri Aug 26 10:51:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 26 10:51:55.666107 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4202b91-1bb8-475d-b5bb-e827ee473968 None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:51:55.668606 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eecb1c4-f250-4055-8f04-9693500fcbdf None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:51:55.769853 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4202b91-1bb8-475d-b5bb-e827ee473968 None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:51:55.770070 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eecb1c4-f250-4055-8f04-9693500fcbdf None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:51:55.804386 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8385ba-abf6-4b03-808c-792181cb527a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:51:55.804719 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9514d47a-e6d9-4e7e-a7a3-71a758ce197a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:51:55.805019 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8385ba-abf6-4b03-808c-792181cb527a None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:51:55.805299 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8385ba-abf6-4b03-808c-792181cb527a None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:51:55.805389 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9514d47a-e6d9-4e7e-a7a3-71a758ce197a None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:51:55.805570 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8385ba-abf6-4b03-808c-792181cb527a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:51:55.805658 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9514d47a-e6d9-4e7e-a7a3-71a758ce197a None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:51:55.805848 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9514d47a-e6d9-4e7e-a7a3-71a758ce197a None admin] PATH_INFO: `/v3/projects/da070dea52de4d71952ed50827d44a1c` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:51:55.807055 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9514d47a-e6d9-4e7e-a7a3-71a758ce197a None admin] RBAC: Authorizing `identity:get_project(project_id=da070dea52de4d71952ed50827d44a1c)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:51:55.808614 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9514d47a-e6d9-4e7e-a7a3-71a758ce197a None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:51:55.810257 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 225/441] 10.222.0.40 () {56 vars in 1186 bytes} [Fri Aug 26 10:51:55 2022] GET /identity/v3/projects/da070dea52de4d71952ed50827d44a1c => generated 298 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 10:51:55.811000 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9514d47a-e6d9-4e7e-a7a3-71a758ce197a None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:51:55.817146 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad8385ba-abf6-4b03-808c-792181cb527a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 10:51:55.821908 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application [None req-ad8385ba-abf6-4b03-808c-792181cb527a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 26 10:51:55.821908 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 10:51:55.821908 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 10:51:55.821908 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 10:51:55.821908 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 10:51:55.821908 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 10:51:55.821908 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 10:51:55.821908 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 10:51:55.821908 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 10:51:55.821908 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 10:51:55.821908 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 10:51:55.821908 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 10:51:55.821908 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 10:51:55.821908 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 10:51:55.821908 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 10:51:55.821908 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 10:51:55.822829 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 10:51:55.822829 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 10:51:55.822829 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 10:51:55.822829 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 10:51:55.822829 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 10:51:55.822829 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 10:51:55.822829 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 10:51:55.822829 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 10:51:55.822829 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 10:51:55.822829 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 10:51:55.822829 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 10:51:55.822829 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application with Lock( Aug 26 10:51:55.822829 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 10:51:55.822829 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._enter() Aug 26 10:51:55.822829 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 10:51:55.822829 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 10:51:55.823957 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 10:51:55.823957 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.creator() Aug 26 10:51:55.823957 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 10:51:55.823957 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application created_value = creator( Aug 26 10:51:55.823957 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 10:51:55.823957 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 10:51:55.823957 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 10:51:55.823957 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 10:51:55.823957 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 10:51:55.823957 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 10:51:55.823957 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 26 10:51:55.823957 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Aug 26 10:51:55.823957 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 217/442] 10.222.0.40 () {56 vars in 1096 bytes} [Fri Aug 26 10:51:55 2022] GET /identity/v3/roles/admin => generated 83 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 10:51:55.823957 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8385ba-abf6-4b03-808c-792181cb527a None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:51:55.847358 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:51:55.887512 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:51:55.914750 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9514d47a-e6d9-4e7e-a7a3-71a758ce197a None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:51:55.928460 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8385ba-abf6-4b03-808c-792181cb527a None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:51:55.948778 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50332874-3d12-480a-8703-c2d45dda7927 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:51:55.949396 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50332874-3d12-480a-8703-c2d45dda7927 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:51:55.949675 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50332874-3d12-480a-8703-c2d45dda7927 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:51:55.949941 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50332874-3d12-480a-8703-c2d45dda7927 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:51:55.960285 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50332874-3d12-480a-8703-c2d45dda7927 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 10:51:55.962426 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29486417-2978-4304-9ee7-527bbb43c9ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:51:55.963041 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29486417-2978-4304-9ee7-527bbb43c9ca None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:51:55.963313 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29486417-2978-4304-9ee7-527bbb43c9ca None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:51:55.963579 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29486417-2978-4304-9ee7-527bbb43c9ca None admin] PATH_INFO: `/v3/groups/9b19097b64bb472a99936052ecdf4232` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:51:55.964685 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29486417-2978-4304-9ee7-527bbb43c9ca None admin] RBAC: Authorizing `identity:get_group(group_id=9b19097b64bb472a99936052ecdf4232)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:51:55.965018 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application [None req-50332874-3d12-480a-8703-c2d45dda7927 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 26 10:51:55.965018 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 10:51:55.965018 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 10:51:55.965018 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 10:51:55.965018 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 10:51:55.965018 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 10:51:55.965018 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 10:51:55.965018 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 10:51:55.965018 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 10:51:55.965018 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 10:51:55.965018 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 10:51:55.965018 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 10:51:55.965018 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 10:51:55.965018 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 10:51:55.965018 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 10:51:55.965018 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 10:51:55.965989 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 10:51:55.965989 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 10:51:55.965989 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 10:51:55.965989 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 10:51:55.965989 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 10:51:55.965989 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 10:51:55.965989 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 10:51:55.965989 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 10:51:55.965989 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 10:51:55.965989 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 10:51:55.965989 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 10:51:55.965989 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application with Lock( Aug 26 10:51:55.965989 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 10:51:55.965989 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._enter() Aug 26 10:51:55.965989 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 10:51:55.965989 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 10:51:55.966807 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 10:51:55.966807 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.creator() Aug 26 10:51:55.966807 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 10:51:55.966807 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application created_value = creator( Aug 26 10:51:55.966807 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 10:51:55.966807 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 10:51:55.966807 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 10:51:55.966807 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 10:51:55.966807 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 10:51:55.966807 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 10:51:55.966807 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 26 10:51:55.966807 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Aug 26 10:51:55.967792 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29486417-2978-4304-9ee7-527bbb43c9ca None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:51:55.967860 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 226/443] 10.222.0.40 () {56 vars in 1114 bytes} [Fri Aug 26 10:51:55 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 10:51:55.967860 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50332874-3d12-480a-8703-c2d45dda7927 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:51:55.968193 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 218/444] 10.222.0.40 () {56 vars in 1180 bytes} [Fri Aug 26 10:51:55 2022] GET /identity/v3/groups/9b19097b64bb472a99936052ecdf4232 => generated 226 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 10:51:55.971893 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29486417-2978-4304-9ee7-527bbb43c9ca None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:51:56.064840 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:51:56.072406 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50332874-3d12-480a-8703-c2d45dda7927 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:51:56.072544 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29486417-2978-4304-9ee7-527bbb43c9ca None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:51:56.097902 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:51:56.106337 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da8495b-53d5-42ae-a419-cb17b9ea59ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:51:56.106537 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c424007-50bd-4844-8ee2-e747a45c3c00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:51:56.106936 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da8495b-53d5-42ae-a419-cb17b9ea59ba None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:51:56.107131 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c424007-50bd-4844-8ee2-e747a45c3c00 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:51:56.107242 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da8495b-53d5-42ae-a419-cb17b9ea59ba None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:51:56.107405 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c424007-50bd-4844-8ee2-e747a45c3c00 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:51:56.107490 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da8495b-53d5-42ae-a419-cb17b9ea59ba None admin] PATH_INFO: `/v3/roles` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:51:56.107667 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c424007-50bd-4844-8ee2-e747a45c3c00 None admin] PATH_INFO: `/v3/roles` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:51:56.108371 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da8495b-53d5-42ae-a419-cb17b9ea59ba None admin] RBAC: Adding query filter params (name=admin) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 10:51:56.108574 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c424007-50bd-4844-8ee2-e747a45c3c00 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 10:51:56.108660 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da8495b-53d5-42ae-a419-cb17b9ea59ba None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:51:56.108929 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c424007-50bd-4844-8ee2-e747a45c3c00 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:51:56.110077 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da8495b-53d5-42ae-a419-cb17b9ea59ba None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:51:56.110332 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c424007-50bd-4844-8ee2-e747a45c3c00 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:51:56.118653 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 227/445] 10.222.0.40 () {56 vars in 1099 bytes} [Fri Aug 26 10:51:55 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 10:51:56.118900 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 219/446] 10.222.0.40 () {56 vars in 1111 bytes} [Fri Aug 26 10:51:55 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 10:51:56.119864 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da8495b-53d5-42ae-a419-cb17b9ea59ba None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:51:56.127986 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c424007-50bd-4844-8ee2-e747a45c3c00 None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:51:56.219026 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da8495b-53d5-42ae-a419-cb17b9ea59ba None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:51:56.226359 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c424007-50bd-4844-8ee2-e747a45c3c00 None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:51:56.254306 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a92a69b-89a8-417e-aed2-ca3d5197c210 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:51:56.254918 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a92a69b-89a8-417e-aed2-ca3d5197c210 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:51:56.255201 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a92a69b-89a8-417e-aed2-ca3d5197c210 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:51:56.255471 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a92a69b-89a8-417e-aed2-ca3d5197c210 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:51:56.256195 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a92a69b-89a8-417e-aed2-ca3d5197c210 None admin] RBAC: Adding query filter params (group.id=9b19097b64bb472a99936052ecdf4232, role.id=9c439f898126444f92abb01910f1d02b, scope.project.id=da070dea52de4d71952ed50827d44a1c) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 10:51:56.256506 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a92a69b-89a8-417e-aed2-ca3d5197c210 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:51:56.257955 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a92a69b-89a8-417e-aed2-ca3d5197c210 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:51:56.260670 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f3048b-c8e0-4b08-954c-47ad0263de7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:51:56.261280 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f3048b-c8e0-4b08-954c-47ad0263de7d None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:51:56.261557 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f3048b-c8e0-4b08-954c-47ad0263de7d None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:51:56.261826 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f3048b-c8e0-4b08-954c-47ad0263de7d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:51:56.263775 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 228/447] 10.222.0.40 () {56 vars in 1376 bytes} [Fri Aug 26 10:51:56 2022] GET /identity/v3/role_assignments?group.id=9b19097b64bb472a99936052ecdf4232&scope.project.id=da070dea52de4d71952ed50827d44a1c&role.id=9c439f898126444f92abb01910f1d02b => generated 611 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 10:51:56.264551 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a92a69b-89a8-417e-aed2-ca3d5197c210 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:51:56.289822 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:51:56.329923 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:51:56.342203 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f3048b-c8e0-4b08-954c-47ad0263de7d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:51:56.343697 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f3048b-c8e0-4b08-954c-47ad0263de7d None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:51:56.381062 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a92a69b-89a8-417e-aed2-ca3d5197c210 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:51:56.422890 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14a5f6e-2e13-43b5-a912-b046996b7e97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:51:56.424128 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14a5f6e-2e13-43b5-a912-b046996b7e97 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:51:56.424331 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14a5f6e-2e13-43b5-a912-b046996b7e97 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:51:56.424598 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14a5f6e-2e13-43b5-a912-b046996b7e97 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:51:56.467551 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 220/448] 10.222.0.40 () {58 vars in 1282 bytes} [Fri Aug 26 10:51:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 26 10:51:56.474121 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f3048b-c8e0-4b08-954c-47ad0263de7d None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:51:56.506674 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a14a5f6e-2e13-43b5-a912-b046996b7e97 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:51:56.508135 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a14a5f6e-2e13-43b5-a912-b046996b7e97 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:51:56.530825 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:51:56.592696 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:51:56.605390 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f3048b-c8e0-4b08-954c-47ad0263de7d None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:51:56.621476 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 229/449] 10.222.0.40 () {58 vars in 1282 bytes} [Fri Aug 26 10:51:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 357 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 26 10:51:56.627325 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14a5f6e-2e13-43b5-a912-b046996b7e97 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:51:56.651980 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a3e602-b249-48b3-a4e9-574cf39c2fcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:51:56.652615 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a3e602-b249-48b3-a4e9-574cf39c2fcd None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:51:56.652894 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a3e602-b249-48b3-a4e9-574cf39c2fcd None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:51:56.653166 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a3e602-b249-48b3-a4e9-574cf39c2fcd None admin] PATH_INFO: `/v3/projects/f3942b05c97e4a1ca048ce3a0379b7a7` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:51:56.654484 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a3e602-b249-48b3-a4e9-574cf39c2fcd None admin] RBAC: Authorizing `identity:get_project(project_id=f3942b05c97e4a1ca048ce3a0379b7a7)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:51:56.656754 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a3e602-b249-48b3-a4e9-574cf39c2fcd None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:51:56.657587 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 221/450] 10.222.0.40 () {56 vars in 1186 bytes} [Fri Aug 26 10:51:56 2022] GET /identity/v3/projects/f3942b05c97e4a1ca048ce3a0379b7a7 => generated 344 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 10:51:56.663675 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a3e602-b249-48b3-a4e9-574cf39c2fcd None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:51:56.742971 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14a5f6e-2e13-43b5-a912-b046996b7e97 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:51:56.766347 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:51:56.782197 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db027d45-c8a5-439c-b141-194b26109ca1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:51:56.783793 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db027d45-c8a5-439c-b141-194b26109ca1 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:51:56.783793 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db027d45-c8a5-439c-b141-194b26109ca1 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:51:56.784033 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db027d45-c8a5-439c-b141-194b26109ca1 None admin] PATH_INFO: `/v3/projects/15667f91e7f6473cb9aab41830d80f5e` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:51:56.785170 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db027d45-c8a5-439c-b141-194b26109ca1 None admin] RBAC: Authorizing `identity:get_project(project_id=15667f91e7f6473cb9aab41830d80f5e)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:51:56.786658 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db027d45-c8a5-439c-b141-194b26109ca1 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:51:56.788290 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 230/451] 10.222.0.40 () {56 vars in 1186 bytes} [Fri Aug 26 10:51:56 2022] GET /identity/v3/projects/15667f91e7f6473cb9aab41830d80f5e => generated 302 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 10:51:56.790285 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a3e602-b249-48b3-a4e9-574cf39c2fcd None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:51:56.793645 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db027d45-c8a5-439c-b141-194b26109ca1 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:51:56.823502 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ecfa5f-4e22-414f-84bc-9ac2f965ed84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:51:56.824133 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ecfa5f-4e22-414f-84bc-9ac2f965ed84 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:51:56.824408 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ecfa5f-4e22-414f-84bc-9ac2f965ed84 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:51:56.824678 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ecfa5f-4e22-414f-84bc-9ac2f965ed84 None admin] PATH_INFO: `/v3/groups/907d2ba571f44f6c8cf2a75be1eda1e0` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:51:56.825787 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ecfa5f-4e22-414f-84bc-9ac2f965ed84 None admin] RBAC: Authorizing `identity:get_group(group_id=907d2ba571f44f6c8cf2a75be1eda1e0)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:51:56.827262 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ecfa5f-4e22-414f-84bc-9ac2f965ed84 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:51:56.828855 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 222/452] 10.222.0.40 () {56 vars in 1180 bytes} [Fri Aug 26 10:51:56 2022] GET /identity/v3/groups/907d2ba571f44f6c8cf2a75be1eda1e0 => generated 229 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 10:51:56.835140 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ecfa5f-4e22-414f-84bc-9ac2f965ed84 None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:51:56.862994 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:51:56.898737 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db027d45-c8a5-439c-b141-194b26109ca1 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:51:56.931794 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6dade67-cd83-4fe4-b32a-8078d1e0bef5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:51:56.932424 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6dade67-cd83-4fe4-b32a-8078d1e0bef5 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:51:56.932704 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6dade67-cd83-4fe4-b32a-8078d1e0bef5 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:51:56.932976 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6dade67-cd83-4fe4-b32a-8078d1e0bef5 None admin] PATH_INFO: `/v3/groups/9b19097b64bb472a99936052ecdf4232` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:51:56.934128 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6dade67-cd83-4fe4-b32a-8078d1e0bef5 None admin] RBAC: Authorizing `identity:get_group(group_id=9b19097b64bb472a99936052ecdf4232)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:51:56.935075 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ecfa5f-4e22-414f-84bc-9ac2f965ed84 None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:51:56.935567 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6dade67-cd83-4fe4-b32a-8078d1e0bef5 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:51:56.937141 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 231/453] 10.222.0.40 () {56 vars in 1180 bytes} [Fri Aug 26 10:51:56 2022] GET /identity/v3/groups/9b19097b64bb472a99936052ecdf4232 => generated 226 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 10:51:56.943036 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6dade67-cd83-4fe4-b32a-8078d1e0bef5 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:51:56.968339 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7c93f0-7237-4f59-9f4e-464644246fa5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:51:56.968941 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7c93f0-7237-4f59-9f4e-464644246fa5 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:51:56.969221 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7c93f0-7237-4f59-9f4e-464644246fa5 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:51:56.969492 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7c93f0-7237-4f59-9f4e-464644246fa5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:51:56.970263 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da7c93f0-7237-4f59-9f4e-464644246fa5 None admin] RBAC: Adding query filter params (group.id=907d2ba571f44f6c8cf2a75be1eda1e0, role.id=1f97e4602e2e4d6498913a81f9ae8d6a, scope.project.id=f3942b05c97e4a1ca048ce3a0379b7a7) {{(pid=81423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 10:51:56.971172 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da7c93f0-7237-4f59-9f4e-464644246fa5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:51:56.972688 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da7c93f0-7237-4f59-9f4e-464644246fa5 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:51:56.978265 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 223/454] 10.222.0.40 () {56 vars in 1376 bytes} [Fri Aug 26 10:51:56 2022] GET /identity/v3/role_assignments?group.id=907d2ba571f44f6c8cf2a75be1eda1e0&scope.project.id=f3942b05c97e4a1ca048ce3a0379b7a7&role.id=1f97e4602e2e4d6498913a81f9ae8d6a => generated 611 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 10:51:57.016362 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:51:57.062899 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6dade67-cd83-4fe4-b32a-8078d1e0bef5 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:51:57.099117 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb13ec7-02ab-4d5c-96df-c5a10ea2dfdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:51:57.099733 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb13ec7-02ab-4d5c-96df-c5a10ea2dfdc None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:51:57.100003 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb13ec7-02ab-4d5c-96df-c5a10ea2dfdc None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:51:57.100268 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb13ec7-02ab-4d5c-96df-c5a10ea2dfdc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:51:57.100997 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb13ec7-02ab-4d5c-96df-c5a10ea2dfdc None admin] RBAC: Adding query filter params (group.id=9b19097b64bb472a99936052ecdf4232, role.id=9c439f898126444f92abb01910f1d02b, scope.project.id=15667f91e7f6473cb9aab41830d80f5e) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 10:51:57.101301 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb13ec7-02ab-4d5c-96df-c5a10ea2dfdc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:51:57.102785 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb13ec7-02ab-4d5c-96df-c5a10ea2dfdc None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:51:57.109298 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 232/455] 10.222.0.40 () {56 vars in 1376 bytes} [Fri Aug 26 10:51:56 2022] GET /identity/v3/role_assignments?group.id=9b19097b64bb472a99936052ecdf4232&scope.project.id=15667f91e7f6473cb9aab41830d80f5e&role.id=9c439f898126444f92abb01910f1d02b => generated 611 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 10:51:59.511573 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951303fb-c840-4e64-9f44-09e6bcb4e975 None None] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:51:59.512083 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951303fb-c840-4e64-9f44-09e6bcb4e975 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:51:59.512159 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951303fb-c840-4e64-9f44-09e6bcb4e975 None None] PATH_INFO: `/` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:51:59.513555 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 224/456] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 26 10:51:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 10:51:59.519923 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9365d7e2-b100-4aef-ac50-6c267c36bb00 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:51:59.520848 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9365d7e2-b100-4aef-ac50-6c267c36bb00 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:51:59.521330 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9365d7e2-b100-4aef-ac50-6c267c36bb00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:51:59.547018 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-9365d7e2-b100-4aef-ac50-6c267c36bb00 None None] MFA Rules not processed for user `ca155a48e2024c32af3fd2e19e5fac11`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 10:51:59.549128 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-9365d7e2-b100-4aef-ac50-6c267c36bb00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 10:51:59.636743 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 233/457] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 26 10:51:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 10:51:59.772361 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c06ccf5-baa8-43ae-b98e-24a3a6bab851 None None] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:51:59.772965 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c06ccf5-baa8-43ae-b98e-24a3a6bab851 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:51:59.773426 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c06ccf5-baa8-43ae-b98e-24a3a6bab851 None None] PATH_INFO: `/` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:51:59.775283 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 225/458] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 26 10:51:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 10:51:59.780565 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a478c7ec-d598-4c2a-886c-fe3915efa9cf None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:51:59.781082 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a478c7ec-d598-4c2a-886c-fe3915efa9cf None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:51:59.781555 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a478c7ec-d598-4c2a-886c-fe3915efa9cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:51:59.810966 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-a478c7ec-d598-4c2a-886c-fe3915efa9cf None None] MFA Rules not processed for user `ca155a48e2024c32af3fd2e19e5fac11`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 10:51:59.812474 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-a478c7ec-d598-4c2a-886c-fe3915efa9cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 10:51:59.882442 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:51:59.912460 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 234/459] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 26 10:51:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 10:51:59.962602 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc276cc1-803b-4a32-8aee-ac67ebde06cf None None] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:51:59.962877 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc276cc1-803b-4a32-8aee-ac67ebde06cf None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:51:59.963311 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc276cc1-803b-4a32-8aee-ac67ebde06cf None None] PATH_INFO: `/` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:51:59.965037 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 226/460] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 26 10:51:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 10:51:59.971368 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355dd026-dd08-4190-8e91-d0b7493287c8 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:51:59.971829 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355dd026-dd08-4190-8e91-d0b7493287c8 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:51:59.972103 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355dd026-dd08-4190-8e91-d0b7493287c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:51:59.998945 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-355dd026-dd08-4190-8e91-d0b7493287c8 None None] MFA Rules not processed for user `ca155a48e2024c32af3fd2e19e5fac11`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 10:52:00.000128 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-355dd026-dd08-4190-8e91-d0b7493287c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 10:52:00.088328 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 235/461] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 26 10:51:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 10:52:00.168617 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc276cc1-803b-4a32-8aee-ac67ebde06cf None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:52:00.279436 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:52:00.296683 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc276cc1-803b-4a32-8aee-ac67ebde06cf None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:52:00.337437 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d58c94-db52-4839-a9c3-59bd9bde3f8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:52:00.338543 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d58c94-db52-4839-a9c3-59bd9bde3f8e None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:00.339021 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d58c94-db52-4839-a9c3-59bd9bde3f8e None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:00.339503 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d58c94-db52-4839-a9c3-59bd9bde3f8e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:00.342837 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355dd026-dd08-4190-8e91-d0b7493287c8 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:52:00.356454 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d58c94-db52-4839-a9c3-59bd9bde3f8e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:52:00.358208 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d58c94-db52-4839-a9c3-59bd9bde3f8e None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:52:00.373302 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application [None req-85d58c94-db52-4839-a9c3-59bd9bde3f8e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 10:52:00.373302 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 10:52:00.373302 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 10:52:00.373302 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 10:52:00.373302 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 10:52:00.373302 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 10:52:00.373302 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 10:52:00.373302 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 10:52:00.373302 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 10:52:00.373302 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 10:52:00.373302 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 10:52:00.373302 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 10:52:00.373302 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 10:52:00.373302 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 10:52:00.373302 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 10:52:00.373302 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 10:52:00.374268 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 10:52:00.374268 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 10:52:00.374268 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 10:52:00.374268 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 10:52:00.374268 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 10:52:00.374268 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 10:52:00.374268 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 10:52:00.374268 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application with Lock( Aug 26 10:52:00.374268 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 10:52:00.374268 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._enter() Aug 26 10:52:00.374268 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 10:52:00.374268 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 10:52:00.374268 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 10:52:00.374268 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.creator() Aug 26 10:52:00.374268 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 10:52:00.374268 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application created_value = creator( Aug 26 10:52:00.374268 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 10:52:00.375110 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 10:52:00.375110 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 10:52:00.375110 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Aug 26 10:52:00.375110 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 227/462] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Aug 26 10:52:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 10:52:00.380589 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d58c94-db52-4839-a9c3-59bd9bde3f8e None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:52:00.485131 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355dd026-dd08-4190-8e91-d0b7493287c8 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:52:00.497203 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:52:00.507119 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d58c94-db52-4839-a9c3-59bd9bde3f8e None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:52:00.523971 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007193d3-955b-4d0e-9966-4017501193c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:52:00.524622 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007193d3-955b-4d0e-9966-4017501193c8 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:00.525102 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007193d3-955b-4d0e-9966-4017501193c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:00.525520 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007193d3-955b-4d0e-9966-4017501193c8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:00.537593 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-007193d3-955b-4d0e-9966-4017501193c8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:52:00.539295 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-007193d3-955b-4d0e-9966-4017501193c8 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:52:00.553349 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c112208a-9336-4823-9f30-af980a20f7b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:52:00.554136 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c112208a-9336-4823-9f30-af980a20f7b6 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:00.554512 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c112208a-9336-4823-9f30-af980a20f7b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:00.554884 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c112208a-9336-4823-9f30-af980a20f7b6 None admin] PATH_INFO: `/v3/domains` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:00.555631 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c112208a-9336-4823-9f30-af980a20f7b6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 10:52:00.556033 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c112208a-9336-4823-9f30-af980a20f7b6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:52:00.556222 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application [None req-007193d3-955b-4d0e-9966-4017501193c8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 10:52:00.556222 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 10:52:00.556222 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 10:52:00.556222 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 10:52:00.556222 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 10:52:00.556222 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 10:52:00.556222 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 10:52:00.556222 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 10:52:00.556222 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 10:52:00.556222 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 10:52:00.556222 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 10:52:00.556222 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 10:52:00.556222 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 10:52:00.556222 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 10:52:00.556222 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 10:52:00.556222 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 10:52:00.557202 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 10:52:00.557202 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 10:52:00.557202 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 10:52:00.557202 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 10:52:00.557202 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 10:52:00.557202 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 10:52:00.557202 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 10:52:00.557202 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application with Lock( Aug 26 10:52:00.557202 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 10:52:00.557202 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._enter() Aug 26 10:52:00.557202 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 10:52:00.557202 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 10:52:00.557202 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 10:52:00.557202 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.creator() Aug 26 10:52:00.557202 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 10:52:00.557202 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application created_value = creator( Aug 26 10:52:00.557202 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 10:52:00.558080 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 10:52:00.558080 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 10:52:00.558080 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Aug 26 10:52:00.558239 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c112208a-9336-4823-9f30-af980a20f7b6 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:52:00.560011 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 236/463] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Aug 26 10:52:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 10:52:00.565763 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007193d3-955b-4d0e-9966-4017501193c8 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:52:00.577506 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 228/464] 10.222.0.40 () {56 vars in 1109 bytes} [Fri Aug 26 10:52:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 10:52:00.578668 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c112208a-9336-4823-9f30-af980a20f7b6 None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:52:00.583690 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:52:00.683874 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007193d3-955b-4d0e-9966-4017501193c8 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:52:00.697716 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c112208a-9336-4823-9f30-af980a20f7b6 None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:52:00.723409 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57092ac-6729-4d8d-8b1b-d75527958b6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:52:00.724050 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57092ac-6729-4d8d-8b1b-d75527958b6e None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:00.724332 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57092ac-6729-4d8d-8b1b-d75527958b6e None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:00.724601 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57092ac-6729-4d8d-8b1b-d75527958b6e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:00.735606 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e52050a-f9c5-407a-bed2-471bb7b30f16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:52:00.736024 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e57092ac-6729-4d8d-8b1b-d75527958b6e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:52:00.736261 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e52050a-f9c5-407a-bed2-471bb7b30f16 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:00.736540 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e52050a-f9c5-407a-bed2-471bb7b30f16 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:00.736810 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e52050a-f9c5-407a-bed2-471bb7b30f16 None admin] PATH_INFO: `/v3/domains` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:00.737456 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e52050a-f9c5-407a-bed2-471bb7b30f16 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 10:52:00.737547 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e57092ac-6729-4d8d-8b1b-d75527958b6e None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:52:00.737755 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e52050a-f9c5-407a-bed2-471bb7b30f16 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:52:00.739208 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e52050a-f9c5-407a-bed2-471bb7b30f16 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:52:00.755453 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 229/465] 10.222.0.40 () {56 vars in 1109 bytes} [Fri Aug 26 10:52:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 10:52:00.755453 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e52050a-f9c5-407a-bed2-471bb7b30f16 None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:52:00.755708 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application [None req-e57092ac-6729-4d8d-8b1b-d75527958b6e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 10:52:00.755708 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 10:52:00.755708 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 10:52:00.755708 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 10:52:00.755708 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 10:52:00.755708 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 10:52:00.755708 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 10:52:00.755708 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 10:52:00.755708 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 10:52:00.755708 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 10:52:00.755708 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 10:52:00.755708 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 10:52:00.755708 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 10:52:00.755708 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 10:52:00.755708 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 10:52:00.755708 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 10:52:00.756524 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 10:52:00.756524 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 10:52:00.756524 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 10:52:00.756524 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 10:52:00.756524 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 10:52:00.756524 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 10:52:00.756524 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 10:52:00.756524 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application with Lock( Aug 26 10:52:00.756524 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 10:52:00.756524 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._enter() Aug 26 10:52:00.756524 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 10:52:00.756524 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 10:52:00.756524 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 10:52:00.756524 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.creator() Aug 26 10:52:00.756524 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 10:52:00.756524 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application created_value = creator( Aug 26 10:52:00.756524 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 10:52:00.757360 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 10:52:00.757360 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 10:52:00.757360 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Aug 26 10:52:00.757360 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 237/466] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Aug 26 10:52:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 10:52:00.757360 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7840fa2f-2cac-4411-b3d1-5385b57f46cb None None] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:00.757360 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7840fa2f-2cac-4411-b3d1-5385b57f46cb None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:00.757360 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7840fa2f-2cac-4411-b3d1-5385b57f46cb None None] PATH_INFO: `/` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:00.757360 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 238/467] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 26 10:52:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 10:52:00.757777 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7840fa2f-2cac-4411-b3d1-5385b57f46cb None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:52:00.777363 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:52:00.829323 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:52:00.877291 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e52050a-f9c5-407a-bed2-471bb7b30f16 None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:52:00.879000 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7840fa2f-2cac-4411-b3d1-5385b57f46cb None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:52:00.917012 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612c8844-d251-4c58-9a4a-5f07fc7ce9d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:52:00.917624 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612c8844-d251-4c58-9a4a-5f07fc7ce9d9 None admin] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:00.917902 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612c8844-d251-4c58-9a4a-5f07fc7ce9d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:00.918198 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612c8844-d251-4c58-9a4a-5f07fc7ce9d9 None admin] PATH_INFO: `/v3/users` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:00.918837 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-612c8844-d251-4c58-9a4a-5f07fc7ce9d9 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:52:00.919857 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d58f625-9097-4715-b8bc-9fb350f64d25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:52:00.920259 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-612c8844-d251-4c58-9a4a-5f07fc7ce9d9 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:52:00.920514 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d58f625-9097-4715-b8bc-9fb350f64d25 None admin] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:00.920790 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d58f625-9097-4715-b8bc-9fb350f64d25 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:00.921078 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d58f625-9097-4715-b8bc-9fb350f64d25 None admin] PATH_INFO: `/v3/users` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:00.921709 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d58f625-9097-4715-b8bc-9fb350f64d25 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:52:00.923180 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d58f625-9097-4715-b8bc-9fb350f64d25 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:52:00.943578 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 230/468] 10.222.0.40 () {60 vars in 1132 bytes} [Fri Aug 26 10:52:00 2022] POST /identity/v3/users => generated 246 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 10:52:00.944727 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb6dd55-74b4-4d5a-a6b0-4c052db776d8 None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:00.945007 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb6dd55-74b4-4d5a-a6b0-4c052db776d8 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:00.945276 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb6dd55-74b4-4d5a-a6b0-4c052db776d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:00.946327 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 239/469] 10.222.0.40 () {60 vars in 1132 bytes} [Fri Aug 26 10:52:00 2022] POST /identity/v3/users => generated 248 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 10:52:00.947489 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d58f625-9097-4715-b8bc-9fb350f64d25 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:52:00.991055 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-8cb6dd55-74b4-4d5a-a6b0-4c052db776d8 None None] MFA Rules not processed for user `ca155a48e2024c32af3fd2e19e5fac11`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 10:52:00.992853 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-8cb6dd55-74b4-4d5a-a6b0-4c052db776d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 10:52:01.076485 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:52:01.122813 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d58f625-9097-4715-b8bc-9fb350f64d25 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:52:01.125263 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 231/470] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 26 10:52:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 10:52:01.127498 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50dd7a54-7340-4cb9-9f2f-d8947f9edd4d None None] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:01.128085 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50dd7a54-7340-4cb9-9f2f-d8947f9edd4d None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:01.128613 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50dd7a54-7340-4cb9-9f2f-d8947f9edd4d None None] PATH_INFO: `/` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:01.130318 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 232/471] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 26 10:52:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 10:52:01.136720 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021b4a38-8566-4225-a261-7d589eb0a0df None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:01.137070 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021b4a38-8566-4225-a261-7d589eb0a0df None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:01.137412 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021b4a38-8566-4225-a261-7d589eb0a0df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:01.158161 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-021b4a38-8566-4225-a261-7d589eb0a0df None None] MFA Rules not processed for user `ca155a48e2024c32af3fd2e19e5fac11`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 10:52:01.159564 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-021b4a38-8566-4225-a261-7d589eb0a0df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 10:52:01.167212 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6be625c-c0ea-4e21-b5d9-230b20ce6dc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:52:01.168022 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6be625c-c0ea-4e21-b5d9-230b20ce6dc9 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:01.168443 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6be625c-c0ea-4e21-b5d9-230b20ce6dc9 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:01.168879 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6be625c-c0ea-4e21-b5d9-230b20ce6dc9 None admin] PATH_INFO: `/v3/domains` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:01.170648 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6be625c-c0ea-4e21-b5d9-230b20ce6dc9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 10:52:01.170984 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6be625c-c0ea-4e21-b5d9-230b20ce6dc9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:52:01.172380 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6be625c-c0ea-4e21-b5d9-230b20ce6dc9 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:52:01.195599 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 240/472] 10.222.0.40 () {56 vars in 1109 bytes} [Fri Aug 26 10:52:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 10:52:01.202901 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6be625c-c0ea-4e21-b5d9-230b20ce6dc9 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:52:01.220366 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:52:01.257958 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 233/473] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 26 10:52:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 26 10:52:01.326071 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6be625c-c0ea-4e21-b5d9-230b20ce6dc9 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:52:01.358190 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:52:01.365377 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5de7e57-8d4c-4381-9264-eb7076a14fed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:52:01.366295 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5de7e57-8d4c-4381-9264-eb7076a14fed None admin] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:01.366767 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5de7e57-8d4c-4381-9264-eb7076a14fed None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:01.367244 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5de7e57-8d4c-4381-9264-eb7076a14fed None admin] PATH_INFO: `/v3/users` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:01.368073 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5de7e57-8d4c-4381-9264-eb7076a14fed None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:52:01.369680 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5de7e57-8d4c-4381-9264-eb7076a14fed None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:52:01.406802 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 241/474] 10.222.0.40 () {60 vars in 1132 bytes} [Fri Aug 26 10:52:01 2022] POST /identity/v3/users => generated 248 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 26 10:52:01.534085 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021b4a38-8566-4225-a261-7d589eb0a0df None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:52:01.580989 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5de7e57-8d4c-4381-9264-eb7076a14fed None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:52:01.679663 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021b4a38-8566-4225-a261-7d589eb0a0df None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:52:01.723683 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5de7e57-8d4c-4381-9264-eb7076a14fed None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:52:01.727496 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2481a4b6-03b1-464a-b199-6523e6b6f81b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:52:01.728387 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2481a4b6-03b1-464a-b199-6523e6b6f81b None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:01.728885 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2481a4b6-03b1-464a-b199-6523e6b6f81b None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:01.729346 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2481a4b6-03b1-464a-b199-6523e6b6f81b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:01.743145 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2481a4b6-03b1-464a-b199-6523e6b6f81b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:52:01.745647 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2481a4b6-03b1-464a-b199-6523e6b6f81b None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:52:01.764086 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application [None req-2481a4b6-03b1-464a-b199-6523e6b6f81b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 10:52:01.764086 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 10:52:01.764086 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 10:52:01.764086 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 10:52:01.764086 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 10:52:01.764086 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 10:52:01.764086 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 10:52:01.764086 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 10:52:01.764086 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 10:52:01.764086 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 10:52:01.764086 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 10:52:01.764086 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 10:52:01.764086 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 10:52:01.764086 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 10:52:01.764086 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 10:52:01.764086 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 10:52:01.765032 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 10:52:01.765032 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 10:52:01.765032 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 10:52:01.765032 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 10:52:01.765032 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 10:52:01.765032 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 10:52:01.765032 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 10:52:01.765032 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application with Lock( Aug 26 10:52:01.765032 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 10:52:01.765032 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._enter() Aug 26 10:52:01.765032 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 10:52:01.765032 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 10:52:01.765032 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 10:52:01.765032 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.creator() Aug 26 10:52:01.765032 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 10:52:01.765032 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application created_value = creator( Aug 26 10:52:01.765032 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 10:52:01.766454 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 10:52:01.766454 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 10:52:01.766454 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Aug 26 10:52:01.770616 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 234/475] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Aug 26 10:52:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 10:52:01.777957 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2481a4b6-03b1-464a-b199-6523e6b6f81b None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:52:01.796189 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809c4998-8c2b-4dba-b293-3e09426163a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:52:01.796775 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809c4998-8c2b-4dba-b293-3e09426163a2 None admin] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:01.797157 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809c4998-8c2b-4dba-b293-3e09426163a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:01.797436 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809c4998-8c2b-4dba-b293-3e09426163a2 None admin] PATH_INFO: `/v3/roles` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:01.798176 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-809c4998-8c2b-4dba-b293-3e09426163a2 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:52:01.800105 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-809c4998-8c2b-4dba-b293-3e09426163a2 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:52:01.820818 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.sql.core [None req-809c4998-8c2b-4dba-b293-3e09426163a2 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 26 10:52:01.820818 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Aug 26 10:52:01.820818 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [parameters: {'id': 'ea66ae9b3afa4ef58bf7f4747b7f1e7f', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Aug 26 10:52:01.820818 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81424) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 26 10:52:01.821870 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:52:01.845650 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application [None req-809c4998-8c2b-4dba-b293-3e09426163a2 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Aug 26 10:52:01.845650 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 10:52:01.845650 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 26 10:52:01.845650 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 26 10:52:01.845650 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 26 10:52:01.845650 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 26 10:52:01.845650 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 26 10:52:01.845650 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application result = self._query(query) Aug 26 10:52:01.845650 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 26 10:52:01.845650 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application conn.query(q) Aug 26 10:52:01.845650 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 26 10:52:01.845650 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 26 10:52:01.845650 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 26 10:52:01.845650 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application result.read() Aug 26 10:52:01.845650 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 26 10:52:01.846662 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 26 10:52:01.846662 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 26 10:52:01.846662 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 26 10:52:01.846662 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 26 10:52:01.846662 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 26 10:52:01.846662 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 26 10:52:01.846662 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 26 10:52:01.846662 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 26 10:52:01.846662 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Aug 26 10:52:01.846662 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 26 10:52:01.846662 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Aug 26 10:52:01.846662 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 10:52:01.846662 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 26 10:52:01.846662 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 26 10:52:01.846662 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Aug 26 10:52:01.846662 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return ref.to_dict() Aug 26 10:52:01.846662 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 26 10:52:01.846662 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application next(self.gen) Aug 26 10:52:01.847587 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 26 10:52:01.847587 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application yield resource Aug 26 10:52:01.847587 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 26 10:52:01.847587 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application next(self.gen) Aug 26 10:52:01.847587 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 26 10:52:01.847587 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application self.session.rollback() Aug 26 10:52:01.847587 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 26 10:52:01.847587 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application self.force_reraise() Aug 26 10:52:01.847587 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 26 10:52:01.847587 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise self.value Aug 26 10:52:01.847587 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 26 10:52:01.847587 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 26 10:52:01.847587 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 26 10:52:01.847587 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application session.commit() Aug 26 10:52:01.847587 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 26 10:52:01.847587 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 26 10:52:01.847587 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 26 10:52:01.848443 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application self._prepare_impl() Aug 26 10:52:01.848443 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 26 10:52:01.848443 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application self.session.flush() Aug 26 10:52:01.848443 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 26 10:52:01.848443 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application self._flush(objects) Aug 26 10:52:01.848443 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 26 10:52:01.848443 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 26 10:52:01.848443 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 26 10:52:01.848443 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application compat.raise_( Aug 26 10:52:01.848443 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 26 10:52:01.848443 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise exception Aug 26 10:52:01.848443 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 26 10:52:01.848443 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application flush_context.execute() Aug 26 10:52:01.848443 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 26 10:52:01.848443 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application rec.execute(self) Aug 26 10:52:01.848443 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 26 10:52:01.848443 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 26 10:52:01.849310 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 26 10:52:01.849310 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 26 10:52:01.849310 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Aug 26 10:52:01.849310 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application c = connection._execute_20( Aug 26 10:52:01.849310 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Aug 26 10:52:01.849310 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 26 10:52:01.849310 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Aug 26 10:52:01.849310 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 26 10:52:01.849310 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Aug 26 10:52:01.849310 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 26 10:52:01.849310 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Aug 26 10:52:01.849310 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 26 10:52:01.849310 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Aug 26 10:52:01.849310 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 26 10:52:01.849310 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 26 10:52:01.849310 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise exception Aug 26 10:52:01.850176 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 26 10:52:01.850176 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 26 10:52:01.850176 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 26 10:52:01.850176 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 26 10:52:01.850176 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 26 10:52:01.850176 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application result = self._query(query) Aug 26 10:52:01.850176 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 26 10:52:01.850176 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application conn.query(q) Aug 26 10:52:01.850176 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 26 10:52:01.850176 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 26 10:52:01.850176 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 26 10:52:01.850176 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application result.read() Aug 26 10:52:01.850176 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 26 10:52:01.850176 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 26 10:52:01.850176 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 26 10:52:01.850176 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 26 10:52:01.850974 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 26 10:52:01.850974 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 26 10:52:01.850974 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 26 10:52:01.850974 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 26 10:52:01.850974 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 26 10:52:01.850974 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Aug 26 10:52:01.850974 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application [parameters: {'id': 'ea66ae9b3afa4ef58bf7f4747b7f1e7f', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Aug 26 10:52:01.850974 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 26 10:52:01.850974 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Aug 26 10:52:01.850974 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 26 10:52:01.850974 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Aug 26 10:52:01.850974 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 10:52:01.850974 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 10:52:01.850974 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 10:52:01.850974 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 10:52:01.850974 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 10:52:01.851829 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 10:52:01.851829 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 10:52:01.851829 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 10:52:01.851829 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 10:52:01.851829 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 10:52:01.851829 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 10:52:01.851829 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Aug 26 10:52:01.851829 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Aug 26 10:52:01.851829 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 10:52:01.851829 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 10:52:01.851829 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Aug 26 10:52:01.851829 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Aug 26 10:52:01.851829 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 26 10:52:01.851829 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 26 10:52:01.851829 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Aug 26 10:52:01.851829 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Aug 26 10:52:01.852729 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 242/476] 10.222.0.40 () {60 vars in 1131 bytes} [Fri Aug 26 10:52:01 2022] POST /identity/v3/roles => generated 144 bytes in 267 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Aug 26 10:52:01.855382 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809c4998-8c2b-4dba-b293-3e09426163a2 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:52:01.961273 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:52:01.980886 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2481a4b6-03b1-464a-b199-6523e6b6f81b None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:52:02.048808 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef55de48-dc6f-46af-8d8d-133cd48c64ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:52:02.050340 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef55de48-dc6f-46af-8d8d-133cd48c64ad None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:02.051821 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef55de48-dc6f-46af-8d8d-133cd48c64ad None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:02.052499 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef55de48-dc6f-46af-8d8d-133cd48c64ad None admin] PATH_INFO: `/v3/domains` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:02.053805 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef55de48-dc6f-46af-8d8d-133cd48c64ad None admin] RBAC: Adding query filter params (name=Default) {{(pid=81423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 10:52:02.054349 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef55de48-dc6f-46af-8d8d-133cd48c64ad None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:52:02.056540 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809c4998-8c2b-4dba-b293-3e09426163a2 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:52:02.058176 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef55de48-dc6f-46af-8d8d-133cd48c64ad None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:52:02.085091 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 235/477] 10.222.0.40 () {56 vars in 1109 bytes} [Fri Aug 26 10:52:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 307 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 10:52:02.095007 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef55de48-dc6f-46af-8d8d-133cd48c64ad None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:52:02.114519 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283fd805-9379-476d-893e-e235ebf86bc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:52:02.115509 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283fd805-9379-476d-893e-e235ebf86bc1 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:02.115981 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283fd805-9379-476d-893e-e235ebf86bc1 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:02.116444 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283fd805-9379-476d-893e-e235ebf86bc1 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:02.128024 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-283fd805-9379-476d-893e-e235ebf86bc1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 10:52:02.133206 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application [None req-283fd805-9379-476d-893e-e235ebf86bc1 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 26 10:52:02.133206 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 10:52:02.133206 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 10:52:02.133206 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 10:52:02.133206 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 10:52:02.133206 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 10:52:02.133206 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 10:52:02.133206 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 10:52:02.133206 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 10:52:02.133206 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 10:52:02.133206 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 10:52:02.133206 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 10:52:02.133206 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 10:52:02.133206 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 10:52:02.133206 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 10:52:02.133206 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 10:52:02.134132 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 10:52:02.134132 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 10:52:02.134132 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 10:52:02.134132 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 10:52:02.134132 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 10:52:02.134132 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 10:52:02.134132 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 10:52:02.134132 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 10:52:02.134132 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 10:52:02.134132 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 10:52:02.134132 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 10:52:02.134132 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application with Lock( Aug 26 10:52:02.134132 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 10:52:02.134132 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._enter() Aug 26 10:52:02.134132 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 10:52:02.134132 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 10:52:02.134937 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 10:52:02.134937 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.creator() Aug 26 10:52:02.134937 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 10:52:02.134937 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application created_value = creator( Aug 26 10:52:02.134937 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 10:52:02.134937 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 10:52:02.134937 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 10:52:02.134937 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 10:52:02.134937 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 10:52:02.134937 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 10:52:02.134937 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 26 10:52:02.134937 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Aug 26 10:52:02.137163 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 243/478] 10.222.0.40 () {56 vars in 1114 bytes} [Fri Aug 26 10:52:01 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 282 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 10:52:02.146948 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283fd805-9379-476d-893e-e235ebf86bc1 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:52:02.206116 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:52:02.215886 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef55de48-dc6f-46af-8d8d-133cd48c64ad None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:52:02.271213 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86ce81e-cb6d-4022-964f-ea0c33bb5474 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:52:02.271838 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86ce81e-cb6d-4022-964f-ea0c33bb5474 None admin] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:02.272116 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86ce81e-cb6d-4022-964f-ea0c33bb5474 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:02.272389 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86ce81e-cb6d-4022-964f-ea0c33bb5474 None admin] PATH_INFO: `/v3/users` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:02.273052 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f86ce81e-cb6d-4022-964f-ea0c33bb5474 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:52:02.274521 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f86ce81e-cb6d-4022-964f-ea0c33bb5474 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:52:02.316110 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 236/479] 10.222.0.40 () {60 vars in 1132 bytes} [Fri Aug 26 10:52:02 2022] POST /identity/v3/users => generated 249 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 26 10:52:02.349616 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283fd805-9379-476d-893e-e235ebf86bc1 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:52:02.363189 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:52:02.397294 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388f892d-8a75-4062-ad19-0171a3c8b629 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:52:02.398327 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388f892d-8a75-4062-ad19-0171a3c8b629 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:02.398804 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388f892d-8a75-4062-ad19-0171a3c8b629 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:02.399270 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388f892d-8a75-4062-ad19-0171a3c8b629 None admin] PATH_INFO: `/v3/roles` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:02.400188 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-388f892d-8a75-4062-ad19-0171a3c8b629 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 10:52:02.400723 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-388f892d-8a75-4062-ad19-0171a3c8b629 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:52:02.402375 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-388f892d-8a75-4062-ad19-0171a3c8b629 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:52:02.421009 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 244/480] 10.222.0.40 () {56 vars in 1111 bytes} [Fri Aug 26 10:52:02 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 10:52:03.368900 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30680f42-450e-4eb5-abd3-b707f8bfd96c None None] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:03.369403 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30680f42-450e-4eb5-abd3-b707f8bfd96c None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:03.369403 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30680f42-450e-4eb5-abd3-b707f8bfd96c None None] PATH_INFO: `/` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:03.371033 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 237/481] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 26 10:52:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 10:52:03.376333 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1adb0b-2cb5-480d-a8fe-79f9b733b909 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:03.377257 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1adb0b-2cb5-480d-a8fe-79f9b733b909 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:03.377456 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1adb0b-2cb5-480d-a8fe-79f9b733b909 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:03.396050 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-2c1adb0b-2cb5-480d-a8fe-79f9b733b909 None None] MFA Rules not processed for user `ca155a48e2024c32af3fd2e19e5fac11`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 10:52:03.397268 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-2c1adb0b-2cb5-480d-a8fe-79f9b733b909 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 10:52:03.490679 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 245/482] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 26 10:52:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 10:52:03.519003 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438d21fc-edd2-45ea-9cde-b2c2d2d4a0bc None None] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:03.519287 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438d21fc-edd2-45ea-9cde-b2c2d2d4a0bc None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:03.519560 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438d21fc-edd2-45ea-9cde-b2c2d2d4a0bc None None] PATH_INFO: `/` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:03.520905 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 238/483] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 26 10:52:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 10:52:03.526095 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b55956-a64a-4659-9c88-e7c61493fe05 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:03.526390 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b55956-a64a-4659-9c88-e7c61493fe05 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:03.526668 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b55956-a64a-4659-9c88-e7c61493fe05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:03.549951 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-60b55956-a64a-4659-9c88-e7c61493fe05 None None] MFA Rules not processed for user `ca155a48e2024c32af3fd2e19e5fac11`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 10:52:03.551187 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-60b55956-a64a-4659-9c88-e7c61493fe05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 10:52:03.621815 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 246/484] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 26 10:52:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 10:52:03.838313 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602ef188-2e6b-4f71-bba0-1db89c495b99 None None] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:03.838638 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602ef188-2e6b-4f71-bba0-1db89c495b99 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:03.838860 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602ef188-2e6b-4f71-bba0-1db89c495b99 None None] PATH_INFO: `/` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:03.840525 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 239/485] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 26 10:52:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 10:52:03.846275 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748b4214-631d-4775-85c7-a47494e6574d None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:03.846553 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748b4214-631d-4775-85c7-a47494e6574d None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:03.846821 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748b4214-631d-4775-85c7-a47494e6574d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:03.853372 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602ef188-2e6b-4f71-bba0-1db89c495b99 None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:52:03.875165 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-748b4214-631d-4775-85c7-a47494e6574d None None] MFA Rules not processed for user `ca155a48e2024c32af3fd2e19e5fac11`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 10:52:03.876298 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-748b4214-631d-4775-85c7-a47494e6574d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 10:52:03.928369 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:52:03.993815 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 247/486] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 26 10:52:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 10:52:03.994693 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748b4214-631d-4775-85c7-a47494e6574d None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:52:04.024540 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602ef188-2e6b-4f71-bba0-1db89c495b99 None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:52:04.090394 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078104c9-44b3-4dea-a4d7-0c5e1b75c82f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:52:04.091030 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078104c9-44b3-4dea-a4d7-0c5e1b75c82f None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:04.091306 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078104c9-44b3-4dea-a4d7-0c5e1b75c82f None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:04.091578 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078104c9-44b3-4dea-a4d7-0c5e1b75c82f None admin] PATH_INFO: `/v3/roles/service` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:04.112123 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-078104c9-44b3-4dea-a4d7-0c5e1b75c82f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 10:52:04.117397 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application [None req-078104c9-44b3-4dea-a4d7-0c5e1b75c82f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 26 10:52:04.117397 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 10:52:04.117397 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 10:52:04.117397 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 10:52:04.117397 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 10:52:04.117397 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 10:52:04.117397 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 10:52:04.117397 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 10:52:04.117397 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 10:52:04.117397 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 10:52:04.117397 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 10:52:04.117397 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 10:52:04.117397 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 10:52:04.117397 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 10:52:04.117397 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 10:52:04.117397 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 10:52:04.118293 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 10:52:04.118293 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 10:52:04.118293 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 10:52:04.118293 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 10:52:04.118293 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 10:52:04.118293 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 10:52:04.118293 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 10:52:04.118293 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 10:52:04.118293 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 10:52:04.118293 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 10:52:04.118293 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 10:52:04.118293 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application with Lock( Aug 26 10:52:04.118293 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 10:52:04.118293 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._enter() Aug 26 10:52:04.118293 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 10:52:04.118293 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 10:52:04.119086 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 10:52:04.119086 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.creator() Aug 26 10:52:04.119086 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 10:52:04.119086 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application created_value = creator( Aug 26 10:52:04.119086 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 10:52:04.119086 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 10:52:04.119086 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 10:52:04.119086 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 10:52:04.119086 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 10:52:04.119086 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 10:52:04.119086 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 26 10:52:04.119086 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Aug 26 10:52:04.119086 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 240/487] 10.222.0.40 () {56 vars in 1102 bytes} [Fri Aug 26 10:52:03 2022] GET /identity/v3/roles/service => generated 85 bytes in 266 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 26 10:52:04.127258 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078104c9-44b3-4dea-a4d7-0c5e1b75c82f None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:52:04.150986 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:52:04.179425 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748b4214-631d-4775-85c7-a47494e6574d None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:52:04.247243 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed08af80-be36-40a0-8860-ec9d7b35cb32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:52:04.247890 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed08af80-be36-40a0-8860-ec9d7b35cb32 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:04.248172 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed08af80-be36-40a0-8860-ec9d7b35cb32 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:04.248447 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed08af80-be36-40a0-8860-ec9d7b35cb32 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:04.266159 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078104c9-44b3-4dea-a4d7-0c5e1b75c82f None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:52:04.274168 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed08af80-be36-40a0-8860-ec9d7b35cb32 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 10:52:04.279406 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application [None req-ed08af80-be36-40a0-8860-ec9d7b35cb32 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 26 10:52:04.279406 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 10:52:04.279406 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 10:52:04.279406 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 10:52:04.279406 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 10:52:04.279406 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 10:52:04.279406 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 10:52:04.279406 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 10:52:04.279406 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 10:52:04.279406 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 10:52:04.279406 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 10:52:04.279406 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 10:52:04.279406 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 10:52:04.279406 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 10:52:04.279406 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 10:52:04.279406 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 10:52:04.280282 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 10:52:04.280282 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 10:52:04.280282 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 10:52:04.280282 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 10:52:04.280282 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 10:52:04.280282 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 10:52:04.280282 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 10:52:04.280282 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 10:52:04.280282 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 10:52:04.280282 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 10:52:04.280282 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 10:52:04.280282 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application with Lock( Aug 26 10:52:04.280282 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 10:52:04.280282 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._enter() Aug 26 10:52:04.280282 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 10:52:04.280282 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 10:52:04.281069 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 10:52:04.281069 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.creator() Aug 26 10:52:04.281069 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 10:52:04.281069 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application created_value = creator( Aug 26 10:52:04.281069 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 10:52:04.281069 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 10:52:04.281069 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 10:52:04.281069 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 10:52:04.281069 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 10:52:04.281069 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 10:52:04.281069 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 26 10:52:04.281069 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Aug 26 10:52:04.285582 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 248/488] 10.222.0.40 () {56 vars in 1102 bytes} [Fri Aug 26 10:52:03 2022] GET /identity/v3/roles/service => generated 85 bytes in 286 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 10:52:04.287024 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed08af80-be36-40a0-8860-ec9d7b35cb32 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:52:04.301417 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a3e2d7-b2ad-4259-b8bf-078454c82279 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:52:04.302051 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a3e2d7-b2ad-4259-b8bf-078454c82279 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:04.302333 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a3e2d7-b2ad-4259-b8bf-078454c82279 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:04.302606 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a3e2d7-b2ad-4259-b8bf-078454c82279 None admin] PATH_INFO: `/v3/roles` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:04.303268 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a3e2d7-b2ad-4259-b8bf-078454c82279 None admin] RBAC: Adding query filter params (name=service) {{(pid=81423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 10:52:04.303572 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a3e2d7-b2ad-4259-b8bf-078454c82279 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:52:04.304959 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a3e2d7-b2ad-4259-b8bf-078454c82279 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:52:04.313685 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 241/489] 10.222.0.40 () {56 vars in 1103 bytes} [Fri Aug 26 10:52:04 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 10:52:04.319930 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a3e2d7-b2ad-4259-b8bf-078454c82279 None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:52:04.334333 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:52:04.402735 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:52:04.420113 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed08af80-be36-40a0-8860-ec9d7b35cb32 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:52:04.441347 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a3e2d7-b2ad-4259-b8bf-078454c82279 None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:52:04.463734 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8173e9-0206-4a98-8694-4b66dca0f79b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:52:04.464830 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8173e9-0206-4a98-8694-4b66dca0f79b None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:04.465246 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8173e9-0206-4a98-8694-4b66dca0f79b None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:04.465827 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8173e9-0206-4a98-8694-4b66dca0f79b None admin] PATH_INFO: `/v3/roles` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:04.466975 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e8173e9-0206-4a98-8694-4b66dca0f79b None admin] RBAC: Adding query filter params (name=service) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 10:52:04.467441 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e8173e9-0206-4a98-8694-4b66dca0f79b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:52:04.469339 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e8173e9-0206-4a98-8694-4b66dca0f79b None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:52:04.477737 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed9b52e-2bb7-4d0b-a8ed-720b2c467764 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:52:04.478525 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed9b52e-2bb7-4d0b-a8ed-720b2c467764 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:04.478944 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed9b52e-2bb7-4d0b-a8ed-720b2c467764 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:04.479313 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed9b52e-2bb7-4d0b-a8ed-720b2c467764 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:04.480055 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 249/490] 10.222.0.40 () {56 vars in 1103 bytes} [Fri Aug 26 10:52:04 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 10:52:04.481017 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8173e9-0206-4a98-8694-4b66dca0f79b None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:52:04.492421 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed9b52e-2bb7-4d0b-a8ed-720b2c467764 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:52:04.493871 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed9b52e-2bb7-4d0b-a8ed-720b2c467764 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:52:04.509968 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application [None req-2ed9b52e-2bb7-4d0b-a8ed-720b2c467764 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 10:52:04.509968 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 10:52:04.509968 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 10:52:04.509968 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 10:52:04.509968 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 10:52:04.509968 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 10:52:04.509968 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 10:52:04.509968 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 10:52:04.509968 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 10:52:04.509968 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 10:52:04.509968 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 10:52:04.509968 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 10:52:04.509968 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 10:52:04.509968 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 10:52:04.509968 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 10:52:04.509968 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 10:52:04.511088 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 10:52:04.511088 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 10:52:04.511088 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 10:52:04.511088 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 10:52:04.511088 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 10:52:04.511088 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 10:52:04.511088 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 10:52:04.511088 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application with Lock( Aug 26 10:52:04.511088 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 10:52:04.511088 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._enter() Aug 26 10:52:04.511088 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 10:52:04.511088 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 10:52:04.511088 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 10:52:04.511088 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.creator() Aug 26 10:52:04.511088 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 10:52:04.511088 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application created_value = creator( Aug 26 10:52:04.511088 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 10:52:04.512115 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 10:52:04.512115 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 10:52:04.512115 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Aug 26 10:52:04.518485 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 242/491] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Aug 26 10:52:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 26 10:52:04.519577 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed9b52e-2bb7-4d0b-a8ed-720b2c467764 None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:52:04.612321 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8173e9-0206-4a98-8694-4b66dca0f79b None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:52:04.628805 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed9b52e-2bb7-4d0b-a8ed-720b2c467764 None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:52:04.646382 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:52:04.650685 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:52:04.658352 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864332aa-2cf7-4f5d-afe5-8b7ea9e018cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:52:04.659739 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864332aa-2cf7-4f5d-afe5-8b7ea9e018cb None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:04.660148 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864332aa-2cf7-4f5d-afe5-8b7ea9e018cb None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:04.660966 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864332aa-2cf7-4f5d-afe5-8b7ea9e018cb None admin] PATH_INFO: `/v3/roles/service` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:04.669273 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656cbbb8-1ee6-4083-8d7b-df699799f457 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:52:04.670142 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656cbbb8-1ee6-4083-8d7b-df699799f457 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:04.670710 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656cbbb8-1ee6-4083-8d7b-df699799f457 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:04.670884 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656cbbb8-1ee6-4083-8d7b-df699799f457 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:04.675474 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-864332aa-2cf7-4f5d-afe5-8b7ea9e018cb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 10:52:04.681346 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application [None req-864332aa-2cf7-4f5d-afe5-8b7ea9e018cb None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 26 10:52:04.681346 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 10:52:04.681346 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 10:52:04.681346 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 10:52:04.681346 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 10:52:04.681346 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 10:52:04.681346 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 10:52:04.681346 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 10:52:04.681346 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 10:52:04.681346 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 10:52:04.681346 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 10:52:04.681346 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 10:52:04.681346 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 10:52:04.681346 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 10:52:04.681346 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 10:52:04.681346 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 10:52:04.682356 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 10:52:04.682356 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 10:52:04.682356 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 10:52:04.682356 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 10:52:04.682356 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 10:52:04.682356 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 10:52:04.682356 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 10:52:04.682356 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 10:52:04.682356 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 10:52:04.682356 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 10:52:04.682356 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 10:52:04.682356 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application with Lock( Aug 26 10:52:04.682356 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 10:52:04.682356 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._enter() Aug 26 10:52:04.682356 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 10:52:04.682356 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 10:52:04.683143 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 10:52:04.683143 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.creator() Aug 26 10:52:04.683143 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 10:52:04.683143 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application created_value = creator( Aug 26 10:52:04.683143 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 10:52:04.683143 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 10:52:04.683143 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 10:52:04.683143 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 10:52:04.683143 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 10:52:04.683143 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 10:52:04.683143 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 26 10:52:04.683143 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Aug 26 10:52:04.685083 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-656cbbb8-1ee6-4083-8d7b-df699799f457 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:52:04.690619 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-656cbbb8-1ee6-4083-8d7b-df699799f457 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:52:04.690787 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 250/492] 10.222.0.40 () {56 vars in 1102 bytes} [Fri Aug 26 10:52:04 2022] GET /identity/v3/roles/service => generated 85 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 10:52:04.691228 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864332aa-2cf7-4f5d-afe5-8b7ea9e018cb None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:52:04.707547 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application [None req-656cbbb8-1ee6-4083-8d7b-df699799f457 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 10:52:04.707547 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 10:52:04.707547 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 10:52:04.707547 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 10:52:04.707547 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 10:52:04.707547 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 10:52:04.707547 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 10:52:04.707547 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 10:52:04.707547 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 10:52:04.707547 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 10:52:04.707547 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 10:52:04.707547 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 10:52:04.707547 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 10:52:04.707547 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 10:52:04.707547 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 10:52:04.707547 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 10:52:04.708487 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 10:52:04.708487 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 10:52:04.708487 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 10:52:04.708487 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 10:52:04.708487 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 10:52:04.708487 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 10:52:04.708487 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 10:52:04.708487 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application with Lock( Aug 26 10:52:04.708487 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 10:52:04.708487 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._enter() Aug 26 10:52:04.708487 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 10:52:04.708487 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 10:52:04.708487 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 10:52:04.708487 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.creator() Aug 26 10:52:04.708487 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 10:52:04.708487 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application created_value = creator( Aug 26 10:52:04.708487 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 10:52:04.709356 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 10:52:04.709356 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 10:52:04.709356 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Aug 26 10:52:04.709356 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 243/493] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Aug 26 10:52:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 10:52:04.710017 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656cbbb8-1ee6-4083-8d7b-df699799f457 None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:52:04.809884 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864332aa-2cf7-4f5d-afe5-8b7ea9e018cb None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:52:04.818804 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656cbbb8-1ee6-4083-8d7b-df699799f457 None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:52:04.845202 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9383b1-1c84-4c50-bc33-26ced9172698 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:52:04.845758 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9383b1-1c84-4c50-bc33-26ced9172698 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:04.846063 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9383b1-1c84-4c50-bc33-26ced9172698 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:04.846335 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9383b1-1c84-4c50-bc33-26ced9172698 None admin] PATH_INFO: `/v3/domains` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:04.846969 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a9383b1-1c84-4c50-bc33-26ced9172698 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 10:52:04.847265 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a9383b1-1c84-4c50-bc33-26ced9172698 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:52:04.848671 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a9383b1-1c84-4c50-bc33-26ced9172698 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:52:04.860029 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df13f67-c013-4aca-bdef-442a1afe423a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:52:04.860641 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df13f67-c013-4aca-bdef-442a1afe423a None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:04.861099 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df13f67-c013-4aca-bdef-442a1afe423a None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:04.861381 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df13f67-c013-4aca-bdef-442a1afe423a None admin] PATH_INFO: `/v3/roles` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:04.862083 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df13f67-c013-4aca-bdef-442a1afe423a None admin] RBAC: Adding query filter params (name=service) {{(pid=81423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 10:52:04.862388 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df13f67-c013-4aca-bdef-442a1afe423a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:52:04.863763 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df13f67-c013-4aca-bdef-442a1afe423a None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:52:04.865382 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 251/494] 10.222.0.40 () {56 vars in 1109 bytes} [Fri Aug 26 10:52:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 10:52:04.865943 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9383b1-1c84-4c50-bc33-26ced9172698 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:52:04.869674 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:52:04.879307 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 244/495] 10.222.0.40 () {56 vars in 1103 bytes} [Fri Aug 26 10:52:04 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 10:52:04.881300 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b9729c-b6dd-4549-9d7e-55dab49fb12d None None] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:04.881687 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b9729c-b6dd-4549-9d7e-55dab49fb12d None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:04.882114 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b9729c-b6dd-4549-9d7e-55dab49fb12d None None] PATH_INFO: `/` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:04.884239 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 245/496] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 26 10:52:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 10:52:04.884958 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b9729c-b6dd-4549-9d7e-55dab49fb12d None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:52:04.923963 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:52:04.973234 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9383b1-1c84-4c50-bc33-26ced9172698 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:52:04.990816 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b9729c-b6dd-4549-9d7e-55dab49fb12d None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:52:05.007336 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c89c67bc-bca6-45b2-b88b-71f57d92c413 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:52:05.007967 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89c67bc-bca6-45b2-b88b-71f57d92c413 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:05.008252 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89c67bc-bca6-45b2-b88b-71f57d92c413 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:05.008526 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89c67bc-bca6-45b2-b88b-71f57d92c413 None admin] PATH_INFO: `/v3/domains` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:05.009161 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c89c67bc-bca6-45b2-b88b-71f57d92c413 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 10:52:05.009464 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c89c67bc-bca6-45b2-b88b-71f57d92c413 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:52:05.010882 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c89c67bc-bca6-45b2-b88b-71f57d92c413 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:52:05.022340 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 252/497] 10.222.0.40 () {56 vars in 1109 bytes} [Fri Aug 26 10:52:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 10:52:05.023129 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c89c67bc-bca6-45b2-b88b-71f57d92c413 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:52:05.024509 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836aff0f-56c6-41ee-82c3-62efa8546984 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:52:05.025133 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836aff0f-56c6-41ee-82c3-62efa8546984 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:05.025415 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836aff0f-56c6-41ee-82c3-62efa8546984 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:05.025687 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836aff0f-56c6-41ee-82c3-62efa8546984 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:05.032985 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-836aff0f-56c6-41ee-82c3-62efa8546984 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:52:05.034474 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-836aff0f-56c6-41ee-82c3-62efa8546984 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:52:05.047542 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application [None req-836aff0f-56c6-41ee-82c3-62efa8546984 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 26 10:52:05.047542 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 10:52:05.047542 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 10:52:05.047542 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 10:52:05.047542 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 10:52:05.047542 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 10:52:05.047542 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 10:52:05.047542 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 10:52:05.047542 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 10:52:05.047542 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 10:52:05.047542 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 10:52:05.047542 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 10:52:05.047542 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 26 10:52:05.047542 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 26 10:52:05.047542 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 26 10:52:05.047542 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 26 10:52:05.049154 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 10:52:05.049154 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 10:52:05.049154 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 26 10:52:05.049154 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 10:52:05.049154 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 26 10:52:05.049154 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 10:52:05.049154 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 10:52:05.049154 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 10:52:05.049154 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 10:52:05.049154 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 10:52:05.049154 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 10:52:05.049154 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application with Lock( Aug 26 10:52:05.049154 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 10:52:05.049154 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._enter() Aug 26 10:52:05.049154 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 10:52:05.049154 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 10:52:05.049154 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 10:52:05.049154 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.creator() Aug 26 10:52:05.050228 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 10:52:05.050228 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application created_value = creator( Aug 26 10:52:05.050228 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 26 10:52:05.050228 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 26 10:52:05.050228 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 26 10:52:05.050228 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 26 10:52:05.050228 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 26 10:52:05.050228 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 26 10:52:05.050228 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 26 10:52:05.050228 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Aug 26 10:52:05.050228 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 246/498] 10.222.0.40 () {56 vars in 1093 bytes} [Fri Aug 26 10:52:04 2022] GET /identity/v3/users/nova => generated 82 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 10:52:05.050784 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27816ad-c25c-4791-bc25-d50e6e6acfff None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:05.050995 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27816ad-c25c-4791-bc25-d50e6e6acfff None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:05.051279 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27816ad-c25c-4791-bc25-d50e6e6acfff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:05.069300 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-f27816ad-c25c-4791-bc25-d50e6e6acfff None None] MFA Rules not processed for user `ca155a48e2024c32af3fd2e19e5fac11`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 10:52:05.070624 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-f27816ad-c25c-4791-bc25-d50e6e6acfff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 10:52:05.088464 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:52:05.129696 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c89c67bc-bca6-45b2-b88b-71f57d92c413 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:52:05.137337 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 247/499] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 26 10:52:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 10:52:05.138612 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f669ecf-4bd2-4339-8916-402a03ffb4ab None None] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:05.138892 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f669ecf-4bd2-4339-8916-402a03ffb4ab None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:05.139160 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f669ecf-4bd2-4339-8916-402a03ffb4ab None None] PATH_INFO: `/` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:05.140585 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 248/500] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 26 10:52:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 10:52:05.141089 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f669ecf-4bd2-4339-8916-402a03ffb4ab None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:52:05.168929 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bed043a-4ce0-4740-9948-95b8fb3cc607 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:52:05.169631 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bed043a-4ce0-4740-9948-95b8fb3cc607 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:05.169904 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bed043a-4ce0-4740-9948-95b8fb3cc607 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:05.170207 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bed043a-4ce0-4740-9948-95b8fb3cc607 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:05.170558 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:52:05.182357 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bed043a-4ce0-4740-9948-95b8fb3cc607 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:52:05.183750 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bed043a-4ce0-4740-9948-95b8fb3cc607 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:52:05.199498 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application [None req-3bed043a-4ce0-4740-9948-95b8fb3cc607 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 10:52:05.199498 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 10:52:05.199498 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 10:52:05.199498 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 10:52:05.199498 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 10:52:05.199498 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 10:52:05.199498 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 10:52:05.199498 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 10:52:05.199498 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 10:52:05.199498 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 10:52:05.199498 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 10:52:05.199498 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 10:52:05.199498 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 10:52:05.199498 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 10:52:05.199498 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 10:52:05.199498 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 10:52:05.200404 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 10:52:05.200404 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 10:52:05.200404 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 10:52:05.200404 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 10:52:05.200404 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 10:52:05.200404 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 10:52:05.200404 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 10:52:05.200404 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application with Lock( Aug 26 10:52:05.200404 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 10:52:05.200404 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._enter() Aug 26 10:52:05.200404 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 10:52:05.200404 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 10:52:05.200404 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 10:52:05.200404 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.creator() Aug 26 10:52:05.200404 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 10:52:05.200404 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application created_value = creator( Aug 26 10:52:05.200404 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 10:52:05.201288 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 10:52:05.201288 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 10:52:05.201288 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Aug 26 10:52:05.201288 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 253/501] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Aug 26 10:52:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 10:52:05.201520 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bed043a-4ce0-4740-9948-95b8fb3cc607 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:52:05.262653 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f669ecf-4bd2-4339-8916-402a03ffb4ab None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:52:05.299471 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e173878-5d6e-4fca-a3cd-adda747f9e4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:52:05.300100 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e173878-5d6e-4fca-a3cd-adda747f9e4e None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:05.300376 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e173878-5d6e-4fca-a3cd-adda747f9e4e None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:05.300643 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e173878-5d6e-4fca-a3cd-adda747f9e4e None admin] PATH_INFO: `/v3/users/glance` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:05.307800 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e173878-5d6e-4fca-a3cd-adda747f9e4e None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:52:05.309221 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e173878-5d6e-4fca-a3cd-adda747f9e4e None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:52:05.310740 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bed043a-4ce0-4740-9948-95b8fb3cc607 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:52:05.321337 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application [None req-4e173878-5d6e-4fca-a3cd-adda747f9e4e None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 26 10:52:05.321337 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 10:52:05.321337 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 10:52:05.321337 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 10:52:05.321337 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 10:52:05.321337 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 10:52:05.321337 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 10:52:05.321337 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 10:52:05.321337 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 10:52:05.321337 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 10:52:05.321337 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 10:52:05.321337 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 10:52:05.321337 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 26 10:52:05.321337 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 26 10:52:05.321337 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 26 10:52:05.321337 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 26 10:52:05.322295 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 10:52:05.322295 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 10:52:05.322295 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 26 10:52:05.322295 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 10:52:05.322295 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 26 10:52:05.322295 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 10:52:05.322295 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 10:52:05.322295 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 10:52:05.322295 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 10:52:05.322295 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 10:52:05.322295 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 10:52:05.322295 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application with Lock( Aug 26 10:52:05.322295 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 10:52:05.322295 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._enter() Aug 26 10:52:05.322295 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 10:52:05.322295 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 10:52:05.322295 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 10:52:05.322295 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.creator() Aug 26 10:52:05.323183 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 10:52:05.323183 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application created_value = creator( Aug 26 10:52:05.323183 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 26 10:52:05.323183 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 26 10:52:05.323183 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 26 10:52:05.323183 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 26 10:52:05.323183 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 26 10:52:05.323183 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 26 10:52:05.323183 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 26 10:52:05.323183 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Aug 26 10:52:05.323183 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 249/502] 10.222.0.40 () {56 vars in 1099 bytes} [Fri Aug 26 10:52:05 2022] GET /identity/v3/users/glance => generated 84 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 10:52:05.324051 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d13af8-18d1-4086-ab29-15399585dfb0 None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:05.324051 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d13af8-18d1-4086-ab29-15399585dfb0 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:05.324174 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d13af8-18d1-4086-ab29-15399585dfb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:05.332911 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:52:05.343240 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-a0d13af8-18d1-4086-ab29-15399585dfb0 None None] MFA Rules not processed for user `ca155a48e2024c32af3fd2e19e5fac11`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 10:52:05.344414 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-a0d13af8-18d1-4086-ab29-15399585dfb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 10:52:05.353239 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81687573-1f13-4299-987f-914edfeb377a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:52:05.353850 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81687573-1f13-4299-987f-914edfeb377a None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:05.354168 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81687573-1f13-4299-987f-914edfeb377a None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:05.354440 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81687573-1f13-4299-987f-914edfeb377a None admin] PATH_INFO: `/v3/users` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:05.355172 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81687573-1f13-4299-987f-914edfeb377a None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 10:52:05.355478 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81687573-1f13-4299-987f-914edfeb377a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:52:05.356913 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81687573-1f13-4299-987f-914edfeb377a None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:52:05.399372 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 254/503] 10.222.0.40 () {56 vars in 1133 bytes} [Fri Aug 26 10:52:05 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 10:52:05.399372 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81687573-1f13-4299-987f-914edfeb377a None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:52:05.421773 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 250/504] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 26 10:52:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 10:52:05.422660 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d13af8-18d1-4086-ab29-15399585dfb0 None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:52:05.446222 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:52:05.502652 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81687573-1f13-4299-987f-914edfeb377a None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:52:05.538434 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d13af8-18d1-4086-ab29-15399585dfb0 None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:52:05.538842 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5510dc0-8610-4817-922f-c0ee15235d15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:52:05.539454 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5510dc0-8610-4817-922f-c0ee15235d15 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:05.539834 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5510dc0-8610-4817-922f-c0ee15235d15 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:05.540029 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5510dc0-8610-4817-922f-c0ee15235d15 None admin] PATH_INFO: `/v3/domains` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:05.540628 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5510dc0-8610-4817-922f-c0ee15235d15 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 10:52:05.540927 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5510dc0-8610-4817-922f-c0ee15235d15 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:52:05.542330 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5510dc0-8610-4817-922f-c0ee15235d15 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:52:05.554279 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 255/505] 10.222.0.40 () {56 vars in 1109 bytes} [Fri Aug 26 10:52:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 10:52:05.559112 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5510dc0-8610-4817-922f-c0ee15235d15 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:52:05.575365 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9aab2bb-085f-4332-b4d2-2f47d3f5f45b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:52:05.575971 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9aab2bb-085f-4332-b4d2-2f47d3f5f45b None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:05.576254 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9aab2bb-085f-4332-b4d2-2f47d3f5f45b None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:05.576552 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9aab2bb-085f-4332-b4d2-2f47d3f5f45b None admin] PATH_INFO: `/v3/users` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:05.577255 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9aab2bb-085f-4332-b4d2-2f47d3f5f45b None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 10:52:05.577562 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9aab2bb-085f-4332-b4d2-2f47d3f5f45b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:52:05.579026 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9aab2bb-085f-4332-b4d2-2f47d3f5f45b None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:52:05.595255 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:52:05.617065 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 251/506] 10.222.0.40 () {56 vars in 1137 bytes} [Fri Aug 26 10:52:05 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 10:52:05.617875 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9aab2bb-085f-4332-b4d2-2f47d3f5f45b None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:52:05.670964 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5510dc0-8610-4817-922f-c0ee15235d15 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:52:05.688642 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:52:05.709625 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b2baa8-f4f9-4726-af17-d3e7c915f928 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:52:05.710551 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b2baa8-f4f9-4726-af17-d3e7c915f928 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:05.710998 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b2baa8-f4f9-4726-af17-d3e7c915f928 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:05.711264 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b2baa8-f4f9-4726-af17-d3e7c915f928 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:05.720003 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9aab2bb-085f-4332-b4d2-2f47d3f5f45b None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:52:05.755044 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ec7c82-2f7f-4b96-bfcc-42aa3a122aee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:52:05.755662 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ec7c82-2f7f-4b96-bfcc-42aa3a122aee None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:05.755939 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ec7c82-2f7f-4b96-bfcc-42aa3a122aee None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:05.756205 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ec7c82-2f7f-4b96-bfcc-42aa3a122aee None admin] PATH_INFO: `/v3/roles/service` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:05.766287 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ec7c82-2f7f-4b96-bfcc-42aa3a122aee None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 10:52:05.771475 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application [None req-61ec7c82-2f7f-4b96-bfcc-42aa3a122aee None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 26 10:52:05.771475 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 10:52:05.771475 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 10:52:05.771475 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 10:52:05.771475 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 10:52:05.771475 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 10:52:05.771475 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 10:52:05.771475 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 10:52:05.771475 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 10:52:05.771475 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 10:52:05.771475 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 10:52:05.771475 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 10:52:05.771475 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 10:52:05.771475 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 10:52:05.771475 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 10:52:05.771475 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 10:52:05.772501 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 10:52:05.772501 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 10:52:05.772501 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 10:52:05.772501 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 10:52:05.772501 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 10:52:05.772501 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 10:52:05.772501 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 10:52:05.772501 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 10:52:05.772501 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 10:52:05.772501 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 10:52:05.772501 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 10:52:05.772501 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application with Lock( Aug 26 10:52:05.772501 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 10:52:05.772501 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._enter() Aug 26 10:52:05.772501 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 10:52:05.772501 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 10:52:05.773312 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 10:52:05.773312 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.creator() Aug 26 10:52:05.773312 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 10:52:05.773312 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application created_value = creator( Aug 26 10:52:05.773312 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 10:52:05.773312 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 10:52:05.773312 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 10:52:05.773312 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 10:52:05.773312 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 10:52:05.773312 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 10:52:05.773312 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 26 10:52:05.773312 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Aug 26 10:52:05.773312 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 252/507] 10.222.0.40 () {56 vars in 1102 bytes} [Fri Aug 26 10:52:05 2022] GET /identity/v3/roles/service => generated 85 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 10:52:05.773964 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ec7c82-2f7f-4b96-bfcc-42aa3a122aee None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:52:05.780705 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b2baa8-f4f9-4726-af17-d3e7c915f928 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:52:05.782197 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b2baa8-f4f9-4726-af17-d3e7c915f928 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:52:05.815571 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:52:05.871354 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ec7c82-2f7f-4b96-bfcc-42aa3a122aee None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:52:05.883899 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 256/508] 10.222.0.40 () {58 vars in 1282 bytes} [Fri Aug 26 10:52:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 26 10:52:05.884740 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b2baa8-f4f9-4726-af17-d3e7c915f928 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:52:05.902486 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:52:05.907870 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae1beb0-370a-4df6-9c9e-31b0df0f3fec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:52:05.908522 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae1beb0-370a-4df6-9c9e-31b0df0f3fec None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:05.908797 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae1beb0-370a-4df6-9c9e-31b0df0f3fec None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:05.909064 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae1beb0-370a-4df6-9c9e-31b0df0f3fec None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:05.916496 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae1beb0-370a-4df6-9c9e-31b0df0f3fec None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:52:05.917930 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae1beb0-370a-4df6-9c9e-31b0df0f3fec None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:52:05.929485 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application [None req-6ae1beb0-370a-4df6-9c9e-31b0df0f3fec None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 26 10:52:05.929485 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 10:52:05.929485 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 10:52:05.929485 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 10:52:05.929485 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 10:52:05.929485 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 10:52:05.929485 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 10:52:05.929485 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 10:52:05.929485 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 10:52:05.929485 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 10:52:05.929485 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 10:52:05.929485 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 10:52:05.929485 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 26 10:52:05.929485 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 26 10:52:05.929485 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 26 10:52:05.929485 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 26 10:52:05.930620 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 10:52:05.930620 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 10:52:05.930620 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 26 10:52:05.930620 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 10:52:05.930620 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 26 10:52:05.930620 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 10:52:05.930620 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 10:52:05.930620 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 10:52:05.930620 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 10:52:05.930620 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 10:52:05.930620 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 10:52:05.930620 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application with Lock( Aug 26 10:52:05.930620 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 10:52:05.930620 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._enter() Aug 26 10:52:05.930620 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 10:52:05.930620 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 10:52:05.930620 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 10:52:05.930620 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.creator() Aug 26 10:52:05.931586 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 10:52:05.931586 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application created_value = creator( Aug 26 10:52:05.931586 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 26 10:52:05.931586 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 26 10:52:05.931586 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 26 10:52:05.931586 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 26 10:52:05.931586 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 26 10:52:05.931586 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 26 10:52:05.931586 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 26 10:52:05.931586 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Aug 26 10:52:05.931586 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 253/509] 10.222.0.40 () {56 vars in 1099 bytes} [Fri Aug 26 10:52:05 2022] GET /identity/v3/users/cinder => generated 84 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 10:52:05.931586 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae1beb0-370a-4df6-9c9e-31b0df0f3fec None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:52:05.983715 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b2baa8-f4f9-4726-af17-d3e7c915f928 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:52:06.018544 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a5f50d-a791-4238-a4a4-0ddb49b5a85a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:52:06.019148 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a5f50d-a791-4238-a4a4-0ddb49b5a85a None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:06.019425 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a5f50d-a791-4238-a4a4-0ddb49b5a85a None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:06.019697 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a5f50d-a791-4238-a4a4-0ddb49b5a85a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:06.022247 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:52:06.028956 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae1beb0-370a-4df6-9c9e-31b0df0f3fec None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:52:06.062343 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89c8943-56e5-4a1c-b316-4b3bbb03a499 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:52:06.062967 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89c8943-56e5-4a1c-b316-4b3bbb03a499 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:06.063243 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89c8943-56e5-4a1c-b316-4b3bbb03a499 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:06.063514 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89c8943-56e5-4a1c-b316-4b3bbb03a499 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:06.073739 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a89c8943-56e5-4a1c-b316-4b3bbb03a499 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:52:06.075238 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a89c8943-56e5-4a1c-b316-4b3bbb03a499 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:52:06.087071 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a5f50d-a791-4238-a4a4-0ddb49b5a85a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:52:06.088515 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a5f50d-a791-4238-a4a4-0ddb49b5a85a None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:52:06.089361 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application [None req-a89c8943-56e5-4a1c-b316-4b3bbb03a499 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 10:52:06.089361 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 10:52:06.089361 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 10:52:06.089361 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 10:52:06.089361 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 10:52:06.089361 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 10:52:06.089361 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 10:52:06.089361 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 10:52:06.089361 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 10:52:06.089361 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 10:52:06.089361 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 10:52:06.089361 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 10:52:06.089361 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 10:52:06.089361 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 10:52:06.089361 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 10:52:06.089361 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 10:52:06.090378 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 10:52:06.090378 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 10:52:06.090378 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 10:52:06.090378 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 10:52:06.090378 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 10:52:06.090378 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 10:52:06.090378 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 10:52:06.090378 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application with Lock( Aug 26 10:52:06.090378 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 10:52:06.090378 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._enter() Aug 26 10:52:06.090378 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 10:52:06.090378 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 10:52:06.090378 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 10:52:06.090378 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.creator() Aug 26 10:52:06.090378 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 10:52:06.090378 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application created_value = creator( Aug 26 10:52:06.090378 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 10:52:06.091203 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 10:52:06.091203 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 10:52:06.091203 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Aug 26 10:52:06.091203 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 254/510] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Aug 26 10:52:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 10:52:06.091203 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89c8943-56e5-4a1c-b316-4b3bbb03a499 None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:52:06.139029 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:52:06.192010 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 257/511] 10.222.0.40 () {58 vars in 1282 bytes} [Fri Aug 26 10:52:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 26 10:52:06.192852 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a5f50d-a791-4238-a4a4-0ddb49b5a85a None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:52:06.194503 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89c8943-56e5-4a1c-b316-4b3bbb03a499 None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:52:06.226994 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:52:06.230784 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76b1e30-0bb2-4d56-b219-ffac41432f81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:52:06.231383 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76b1e30-0bb2-4d56-b219-ffac41432f81 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:06.231656 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76b1e30-0bb2-4d56-b219-ffac41432f81 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:06.231922 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76b1e30-0bb2-4d56-b219-ffac41432f81 None admin] PATH_INFO: `/v3/roles` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:06.232569 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e76b1e30-0bb2-4d56-b219-ffac41432f81 None admin] RBAC: Adding query filter params (name=service) {{(pid=81423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 10:52:06.232864 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e76b1e30-0bb2-4d56-b219-ffac41432f81 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:52:06.234568 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e76b1e30-0bb2-4d56-b219-ffac41432f81 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:52:06.243554 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 255/512] 10.222.0.40 () {56 vars in 1103 bytes} [Fri Aug 26 10:52:06 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 10:52:06.244418 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76b1e30-0bb2-4d56-b219-ffac41432f81 None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:52:06.297476 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a5f50d-a791-4238-a4a4-0ddb49b5a85a None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:52:06.331785 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4244ef3-2316-432f-bf9f-bfb09ec237f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:52:06.332448 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4244ef3-2316-432f-bf9f-bfb09ec237f4 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:06.332707 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4244ef3-2316-432f-bf9f-bfb09ec237f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:06.332977 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4244ef3-2316-432f-bf9f-bfb09ec237f4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:06.342869 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76b1e30-0bb2-4d56-b219-ffac41432f81 None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:52:06.346303 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4244ef3-2316-432f-bf9f-bfb09ec237f4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:52:06.347794 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4244ef3-2316-432f-bf9f-bfb09ec237f4 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:52:06.350718 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:52:06.366338 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application [None req-b4244ef3-2316-432f-bf9f-bfb09ec237f4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 10:52:06.366338 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 10:52:06.366338 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 10:52:06.366338 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 10:52:06.366338 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 10:52:06.366338 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 10:52:06.366338 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 10:52:06.366338 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 10:52:06.366338 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 10:52:06.366338 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 10:52:06.366338 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 10:52:06.366338 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 10:52:06.366338 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 10:52:06.366338 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 10:52:06.366338 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 10:52:06.366338 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 10:52:06.367477 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 10:52:06.367477 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 10:52:06.367477 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 10:52:06.367477 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 10:52:06.367477 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 10:52:06.367477 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 10:52:06.367477 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 10:52:06.367477 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application with Lock( Aug 26 10:52:06.367477 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 10:52:06.367477 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._enter() Aug 26 10:52:06.367477 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 10:52:06.367477 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 10:52:06.367477 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 10:52:06.367477 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.creator() Aug 26 10:52:06.367477 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 10:52:06.367477 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application created_value = creator( Aug 26 10:52:06.367477 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 10:52:06.368499 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 10:52:06.368499 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 10:52:06.368499 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Aug 26 10:52:06.368499 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 258/513] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Aug 26 10:52:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 10:52:06.368499 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4244ef3-2316-432f-bf9f-bfb09ec237f4 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:52:06.382344 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6f9f6b-a594-48d9-9919-bb0bbdf52ed8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:52:06.382978 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6f9f6b-a594-48d9-9919-bb0bbdf52ed8 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:06.383255 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6f9f6b-a594-48d9-9919-bb0bbdf52ed8 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:06.383523 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6f9f6b-a594-48d9-9919-bb0bbdf52ed8 None admin] PATH_INFO: `/v3/users` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:06.384248 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e6f9f6b-a594-48d9-9919-bb0bbdf52ed8 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 10:52:06.384547 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e6f9f6b-a594-48d9-9919-bb0bbdf52ed8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:52:06.386071 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e6f9f6b-a594-48d9-9919-bb0bbdf52ed8 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:52:06.402687 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 256/514] 10.222.0.40 () {56 vars in 1137 bytes} [Fri Aug 26 10:52:06 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 10:52:06.403479 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6f9f6b-a594-48d9-9919-bb0bbdf52ed8 None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:52:06.455130 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:52:06.469877 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4244ef3-2316-432f-bf9f-bfb09ec237f4 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:52:06.502623 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6f9f6b-a594-48d9-9919-bb0bbdf52ed8 None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:52:06.505204 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1caae9-1192-4b49-8bb8-e0168b332c53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:52:06.505799 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1caae9-1192-4b49-8bb8-e0168b332c53 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:06.506101 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1caae9-1192-4b49-8bb8-e0168b332c53 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:06.506368 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1caae9-1192-4b49-8bb8-e0168b332c53 None admin] PATH_INFO: `/v3/domains` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:06.506996 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f1caae9-1192-4b49-8bb8-e0168b332c53 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 10:52:06.507293 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f1caae9-1192-4b49-8bb8-e0168b332c53 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:52:06.508661 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f1caae9-1192-4b49-8bb8-e0168b332c53 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:52:06.520327 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 259/515] 10.222.0.40 () {56 vars in 1109 bytes} [Fri Aug 26 10:52:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 10:52:06.521104 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1caae9-1192-4b49-8bb8-e0168b332c53 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:52:06.537159 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0566e7-1662-4561-b2c7-3115576ad24c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:52:06.537783 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0566e7-1662-4561-b2c7-3115576ad24c None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:06.538095 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0566e7-1662-4561-b2c7-3115576ad24c None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:06.538392 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0566e7-1662-4561-b2c7-3115576ad24c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:06.548225 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb0566e7-1662-4561-b2c7-3115576ad24c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:52:06.549706 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb0566e7-1662-4561-b2c7-3115576ad24c None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:52:06.563777 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application [None req-cb0566e7-1662-4561-b2c7-3115576ad24c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 10:52:06.563777 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 10:52:06.563777 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 10:52:06.563777 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 10:52:06.563777 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 10:52:06.563777 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 10:52:06.563777 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 10:52:06.563777 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 10:52:06.563777 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 10:52:06.563777 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 10:52:06.563777 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 10:52:06.563777 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 10:52:06.563777 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 10:52:06.563777 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 10:52:06.563777 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 10:52:06.563777 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 10:52:06.564803 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 10:52:06.564803 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 10:52:06.564803 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 10:52:06.564803 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 10:52:06.564803 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 10:52:06.564803 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 10:52:06.564803 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 10:52:06.564803 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application with Lock( Aug 26 10:52:06.564803 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 10:52:06.564803 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._enter() Aug 26 10:52:06.564803 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 10:52:06.564803 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 10:52:06.564803 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 10:52:06.564803 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.creator() Aug 26 10:52:06.564803 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 10:52:06.564803 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application created_value = creator( Aug 26 10:52:06.564803 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 10:52:06.565656 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 10:52:06.565656 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 10:52:06.565656 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Aug 26 10:52:06.565656 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 257/516] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Aug 26 10:52:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 10:52:06.565867 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0566e7-1662-4561-b2c7-3115576ad24c None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:52:06.584695 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:52:06.623758 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1caae9-1192-4b49-8bb8-e0168b332c53 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:52:06.656608 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d05f7d-f8df-4291-873e-66bc12849b4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:52:06.657231 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d05f7d-f8df-4291-873e-66bc12849b4b None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:06.657507 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d05f7d-f8df-4291-873e-66bc12849b4b None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:06.657910 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d05f7d-f8df-4291-873e-66bc12849b4b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:06.668160 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d05f7d-f8df-4291-873e-66bc12849b4b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:52:06.668386 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0566e7-1662-4561-b2c7-3115576ad24c None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:52:06.669597 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d05f7d-f8df-4291-873e-66bc12849b4b None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:52:06.676279 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:52:06.684572 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application [None req-b8d05f7d-f8df-4291-873e-66bc12849b4b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 10:52:06.684572 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 10:52:06.684572 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 10:52:06.684572 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 10:52:06.684572 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 10:52:06.684572 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 10:52:06.684572 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 10:52:06.684572 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 10:52:06.684572 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 10:52:06.684572 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 10:52:06.684572 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 10:52:06.684572 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 10:52:06.684572 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 10:52:06.684572 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 10:52:06.684572 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 10:52:06.684572 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 10:52:06.685472 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 10:52:06.685472 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 10:52:06.685472 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 10:52:06.685472 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 10:52:06.685472 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 10:52:06.685472 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 10:52:06.685472 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 10:52:06.685472 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application with Lock( Aug 26 10:52:06.685472 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 10:52:06.685472 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._enter() Aug 26 10:52:06.685472 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 10:52:06.685472 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 10:52:06.685472 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 10:52:06.685472 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.creator() Aug 26 10:52:06.685472 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 10:52:06.685472 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application created_value = creator( Aug 26 10:52:06.685472 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 10:52:06.686341 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 10:52:06.686341 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 10:52:06.686341 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Aug 26 10:52:06.686341 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 260/517] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Aug 26 10:52:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 10:52:06.686553 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d05f7d-f8df-4291-873e-66bc12849b4b None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:52:06.706961 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44820e4b-1f30-4bed-aad8-0438f0af0854 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:52:06.707572 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44820e4b-1f30-4bed-aad8-0438f0af0854 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:06.707851 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44820e4b-1f30-4bed-aad8-0438f0af0854 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:06.708119 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44820e4b-1f30-4bed-aad8-0438f0af0854 None admin] PATH_INFO: `/v3/domains` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:06.708756 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44820e4b-1f30-4bed-aad8-0438f0af0854 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 10:52:06.709057 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44820e4b-1f30-4bed-aad8-0438f0af0854 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:52:06.710458 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44820e4b-1f30-4bed-aad8-0438f0af0854 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:52:06.721466 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 258/518] 10.222.0.40 () {56 vars in 1109 bytes} [Fri Aug 26 10:52:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 10:52:06.722266 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44820e4b-1f30-4bed-aad8-0438f0af0854 None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:52:06.788714 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d05f7d-f8df-4291-873e-66bc12849b4b None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:52:06.801718 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:52:06.822017 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bec0501-5e6b-4e47-8479-98b668fe244a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:52:06.822624 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bec0501-5e6b-4e47-8479-98b668fe244a None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:06.822894 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bec0501-5e6b-4e47-8479-98b668fe244a None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:06.823165 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bec0501-5e6b-4e47-8479-98b668fe244a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:06.825869 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44820e4b-1f30-4bed-aad8-0438f0af0854 None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:52:06.859240 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ec6935-fd74-4d22-a5e9-262c7762ecec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:52:06.859841 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ec6935-fd74-4d22-a5e9-262c7762ecec None admin] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:06.860117 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ec6935-fd74-4d22-a5e9-262c7762ecec None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:06.860385 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ec6935-fd74-4d22-a5e9-262c7762ecec None admin] PATH_INFO: `/v3/users` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:06.861586 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ec6935-fd74-4d22-a5e9-262c7762ecec None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:52:06.863048 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ec6935-fd74-4d22-a5e9-262c7762ecec None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:52:06.884848 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 259/519] 10.222.0.40 () {60 vars in 1132 bytes} [Fri Aug 26 10:52:06 2022] POST /identity/v3/users => generated 247 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 10:52:06.885663 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ec6935-fd74-4d22-a5e9-262c7762ecec None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:52:06.888717 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:52:06.892310 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bec0501-5e6b-4e47-8479-98b668fe244a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:52:06.895220 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bec0501-5e6b-4e47-8479-98b668fe244a None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:52:07.006982 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ec6935-fd74-4d22-a5e9-262c7762ecec None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:52:07.010051 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 261/520] 10.222.0.40 () {58 vars in 1282 bytes} [Fri Aug 26 10:52:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 26 10:52:07.010501 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bec0501-5e6b-4e47-8479-98b668fe244a None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:52:07.044908 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:52:07.051515 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6e1923-aefa-426b-acb8-ff2434825855 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:52:07.052488 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6e1923-aefa-426b-acb8-ff2434825855 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:07.052919 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6e1923-aefa-426b-acb8-ff2434825855 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:07.053194 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6e1923-aefa-426b-acb8-ff2434825855 None admin] PATH_INFO: `/v3/domains` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:07.053851 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f6e1923-aefa-426b-acb8-ff2434825855 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 10:52:07.054378 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f6e1923-aefa-426b-acb8-ff2434825855 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:52:07.055765 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f6e1923-aefa-426b-acb8-ff2434825855 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:52:07.072595 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 260/521] 10.222.0.40 () {56 vars in 1109 bytes} [Fri Aug 26 10:52:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 10:52:07.073594 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6e1923-aefa-426b-acb8-ff2434825855 None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:52:07.125497 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:52:07.131251 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bec0501-5e6b-4e47-8479-98b668fe244a None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:52:07.168304 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff6a93e-6975-49cd-9a2b-cdd1d5262b9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:52:07.168904 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff6a93e-6975-49cd-9a2b-cdd1d5262b9b None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:07.169253 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff6a93e-6975-49cd-9a2b-cdd1d5262b9b None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:07.169585 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff6a93e-6975-49cd-9a2b-cdd1d5262b9b None admin] PATH_INFO: `/v3/domains` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:07.170310 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff6a93e-6975-49cd-9a2b-cdd1d5262b9b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 10:52:07.170616 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff6a93e-6975-49cd-9a2b-cdd1d5262b9b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:52:07.172158 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff6a93e-6975-49cd-9a2b-cdd1d5262b9b None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:52:07.184631 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 262/522] 10.222.0.40 () {56 vars in 1109 bytes} [Fri Aug 26 10:52:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 10:52:07.185854 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff6a93e-6975-49cd-9a2b-cdd1d5262b9b None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:52:07.192730 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6e1923-aefa-426b-acb8-ff2434825855 None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:52:07.231328 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1768b3-0112-4e54-9d86-d1d018004d21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:52:07.232369 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1768b3-0112-4e54-9d86-d1d018004d21 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:07.232628 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1768b3-0112-4e54-9d86-d1d018004d21 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:07.232819 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1768b3-0112-4e54-9d86-d1d018004d21 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:07.240774 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f1768b3-0112-4e54-9d86-d1d018004d21 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:52:07.242245 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f1768b3-0112-4e54-9d86-d1d018004d21 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:52:07.254956 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application [None req-2f1768b3-0112-4e54-9d86-d1d018004d21 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 10:52:07.254956 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 10:52:07.254956 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 10:52:07.254956 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 10:52:07.254956 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 10:52:07.254956 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 10:52:07.254956 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 10:52:07.254956 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 10:52:07.254956 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 10:52:07.254956 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 10:52:07.254956 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 10:52:07.254956 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 10:52:07.254956 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 26 10:52:07.254956 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 26 10:52:07.254956 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 26 10:52:07.255858 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 26 10:52:07.255858 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 10:52:07.255858 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 10:52:07.255858 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 10:52:07.255858 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 10:52:07.255858 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 10:52:07.255858 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 10:52:07.255858 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 10:52:07.255858 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application with Lock( Aug 26 10:52:07.255858 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 10:52:07.255858 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._enter() Aug 26 10:52:07.255858 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 10:52:07.255858 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 10:52:07.255858 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 10:52:07.255858 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.creator() Aug 26 10:52:07.255858 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 10:52:07.255858 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application created_value = creator( Aug 26 10:52:07.256707 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 26 10:52:07.256707 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 26 10:52:07.256707 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 26 10:52:07.256707 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 26 10:52:07.256707 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 26 10:52:07.256707 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 26 10:52:07.256707 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 10:52:07.256707 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Aug 26 10:52:07.256707 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 261/523] 10.222.0.40 () {56 vars in 1111 bytes} [Fri Aug 26 10:52:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 10:52:07.257609 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1768b3-0112-4e54-9d86-d1d018004d21 None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:52:07.303377 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff6a93e-6975-49cd-9a2b-cdd1d5262b9b None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:52:07.316382 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:52:07.338432 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c15d141-72eb-402a-a16e-f8f154d3aa4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:52:07.339062 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c15d141-72eb-402a-a16e-f8f154d3aa4e None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:07.339339 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c15d141-72eb-402a-a16e-f8f154d3aa4e None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:07.339607 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c15d141-72eb-402a-a16e-f8f154d3aa4e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:07.350382 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c15d141-72eb-402a-a16e-f8f154d3aa4e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:52:07.351838 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c15d141-72eb-402a-a16e-f8f154d3aa4e None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:52:07.367159 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application [None req-3c15d141-72eb-402a-a16e-f8f154d3aa4e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 10:52:07.367159 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 10:52:07.367159 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 10:52:07.367159 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 10:52:07.367159 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 10:52:07.367159 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 10:52:07.367159 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 10:52:07.367159 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 10:52:07.367159 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 10:52:07.367159 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 10:52:07.367159 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 10:52:07.367159 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 10:52:07.367159 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 10:52:07.367159 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 10:52:07.367159 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 10:52:07.367159 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 10:52:07.368238 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 10:52:07.368238 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 10:52:07.368238 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 10:52:07.368238 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 10:52:07.368238 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 10:52:07.368238 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 10:52:07.368238 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 10:52:07.368238 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application with Lock( Aug 26 10:52:07.368238 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 10:52:07.368238 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._enter() Aug 26 10:52:07.368238 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 10:52:07.368238 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 10:52:07.368238 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 10:52:07.368238 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.creator() Aug 26 10:52:07.368238 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 10:52:07.368238 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application created_value = creator( Aug 26 10:52:07.368238 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 10:52:07.369119 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 10:52:07.369119 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 10:52:07.369119 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Aug 26 10:52:07.369283 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1768b3-0112-4e54-9d86-d1d018004d21 None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:52:07.373363 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 263/524] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Aug 26 10:52:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 10:52:07.373857 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c15d141-72eb-402a-a16e-f8f154d3aa4e None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:52:07.387131 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:52:07.406934 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7516f3cd-c41f-4602-868c-75380409b7f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:52:07.407607 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7516f3cd-c41f-4602-868c-75380409b7f4 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:07.407902 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7516f3cd-c41f-4602-868c-75380409b7f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:07.408276 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7516f3cd-c41f-4602-868c-75380409b7f4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:07.415306 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7516f3cd-c41f-4602-868c-75380409b7f4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:52:07.416756 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7516f3cd-c41f-4602-868c-75380409b7f4 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:52:07.429431 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application [None req-7516f3cd-c41f-4602-868c-75380409b7f4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 10:52:07.429431 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 10:52:07.429431 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 10:52:07.429431 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 10:52:07.429431 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 10:52:07.429431 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 10:52:07.429431 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 10:52:07.429431 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 10:52:07.429431 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 10:52:07.429431 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 10:52:07.429431 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 10:52:07.429431 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 10:52:07.429431 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 26 10:52:07.429431 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 26 10:52:07.429431 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 26 10:52:07.430362 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 26 10:52:07.430362 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 10:52:07.430362 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 10:52:07.430362 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 10:52:07.430362 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 10:52:07.430362 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 10:52:07.430362 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 10:52:07.430362 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 10:52:07.430362 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application with Lock( Aug 26 10:52:07.430362 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 10:52:07.430362 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._enter() Aug 26 10:52:07.430362 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 10:52:07.430362 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 10:52:07.430362 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 10:52:07.430362 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.creator() Aug 26 10:52:07.430362 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 10:52:07.430362 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application created_value = creator( Aug 26 10:52:07.431209 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 26 10:52:07.431209 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 26 10:52:07.431209 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 26 10:52:07.431209 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 26 10:52:07.431209 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 26 10:52:07.431209 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 26 10:52:07.431209 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 10:52:07.431209 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Aug 26 10:52:07.433148 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 262/525] 10.222.0.40 () {56 vars in 1111 bytes} [Fri Aug 26 10:52:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 10:52:07.434115 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7516f3cd-c41f-4602-868c-75380409b7f4 None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:52:07.485610 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c15d141-72eb-402a-a16e-f8f154d3aa4e None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:52:07.521694 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe273178-5a52-4aac-9115-848239a3f9a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:52:07.522393 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe273178-5a52-4aac-9115-848239a3f9a1 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:07.522666 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe273178-5a52-4aac-9115-848239a3f9a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:07.522935 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe273178-5a52-4aac-9115-848239a3f9a1 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:07.531167 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe273178-5a52-4aac-9115-848239a3f9a1 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:52:07.533408 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe273178-5a52-4aac-9115-848239a3f9a1 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:52:07.546165 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application [None req-fe273178-5a52-4aac-9115-848239a3f9a1 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 26 10:52:07.546165 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 10:52:07.546165 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 10:52:07.546165 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 10:52:07.546165 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 10:52:07.546165 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 10:52:07.546165 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 10:52:07.546165 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 10:52:07.546165 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 10:52:07.546165 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 10:52:07.546165 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 10:52:07.546165 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 10:52:07.546165 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 26 10:52:07.546165 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 26 10:52:07.546165 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 26 10:52:07.546165 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 26 10:52:07.547096 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 10:52:07.547096 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 10:52:07.547096 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 26 10:52:07.547096 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 10:52:07.547096 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 26 10:52:07.547096 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 10:52:07.547096 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 10:52:07.547096 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 10:52:07.547096 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 10:52:07.547096 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 10:52:07.547096 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 10:52:07.547096 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application with Lock( Aug 26 10:52:07.547096 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 10:52:07.547096 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._enter() Aug 26 10:52:07.547096 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 10:52:07.547096 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 10:52:07.547096 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 10:52:07.547096 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.creator() Aug 26 10:52:07.547983 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 10:52:07.547983 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application created_value = creator( Aug 26 10:52:07.547983 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 26 10:52:07.547983 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 26 10:52:07.547983 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 26 10:52:07.547983 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 26 10:52:07.547983 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 26 10:52:07.547983 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 26 10:52:07.547983 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 26 10:52:07.547983 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Aug 26 10:52:07.547983 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 264/526] 10.222.0.40 () {56 vars in 1102 bytes} [Fri Aug 26 10:52:07 2022] GET /identity/v3/users/neutron => generated 85 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 10:52:07.548760 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe273178-5a52-4aac-9115-848239a3f9a1 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:52:07.555874 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7516f3cd-c41f-4602-868c-75380409b7f4 None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:52:07.570217 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:52:07.593377 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2cf95fb-35a2-4d1f-aa1a-444603697ad0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:52:07.593988 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cf95fb-35a2-4d1f-aa1a-444603697ad0 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:07.594290 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cf95fb-35a2-4d1f-aa1a-444603697ad0 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:07.594560 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cf95fb-35a2-4d1f-aa1a-444603697ad0 None admin] PATH_INFO: `/v3/projects` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:07.595395 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2cf95fb-35a2-4d1f-aa1a-444603697ad0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 10:52:07.595694 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2cf95fb-35a2-4d1f-aa1a-444603697ad0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:52:07.597115 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2cf95fb-35a2-4d1f-aa1a-444603697ad0 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:52:07.615044 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 263/527] 10.222.0.40 () {56 vars in 1148 bytes} [Fri Aug 26 10:52:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 10:52:07.615845 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2cf95fb-35a2-4d1f-aa1a-444603697ad0 None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:52:07.629936 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:52:07.665899 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe273178-5a52-4aac-9115-848239a3f9a1 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:52:07.701998 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4f089d-f227-41a4-b26b-5fce3bcce2ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:52:07.702845 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4f089d-f227-41a4-b26b-5fce3bcce2ba None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:07.703330 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4f089d-f227-41a4-b26b-5fce3bcce2ba None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:07.703790 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4f089d-f227-41a4-b26b-5fce3bcce2ba None admin] PATH_INFO: `/v3/domains` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:07.705156 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee4f089d-f227-41a4-b26b-5fce3bcce2ba None admin] RBAC: Adding query filter params (name=Default) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 10:52:07.705530 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee4f089d-f227-41a4-b26b-5fce3bcce2ba None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:52:07.706939 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee4f089d-f227-41a4-b26b-5fce3bcce2ba None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:52:07.718935 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 265/528] 10.222.0.40 () {56 vars in 1109 bytes} [Fri Aug 26 10:52:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 10:52:07.719720 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4f089d-f227-41a4-b26b-5fce3bcce2ba None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:52:07.726202 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2cf95fb-35a2-4d1f-aa1a-444603697ad0 None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:52:07.764021 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d21e5e-26ab-4256-80c5-83c72ea5c13f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:52:07.764932 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d21e5e-26ab-4256-80c5-83c72ea5c13f None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:07.765217 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d21e5e-26ab-4256-80c5-83c72ea5c13f None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:07.765635 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d21e5e-26ab-4256-80c5-83c72ea5c13f None admin] PATH_INFO: `/v3/projects` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:07.766364 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d21e5e-26ab-4256-80c5-83c72ea5c13f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 10:52:07.766671 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d21e5e-26ab-4256-80c5-83c72ea5c13f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:52:07.769054 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d21e5e-26ab-4256-80c5-83c72ea5c13f None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:52:07.785869 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 264/529] 10.222.0.40 () {56 vars in 1148 bytes} [Fri Aug 26 10:52:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 10:52:07.786841 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d21e5e-26ab-4256-80c5-83c72ea5c13f None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:52:07.819364 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:52:07.828025 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4f089d-f227-41a4-b26b-5fce3bcce2ba None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:52:07.858706 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:52:07.866334 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d5d2bc3-e049-4fab-9f34-85cc578c8cd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:52:07.866942 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5d2bc3-e049-4fab-9f34-85cc578c8cd5 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:07.867215 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5d2bc3-e049-4fab-9f34-85cc578c8cd5 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:07.867483 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5d2bc3-e049-4fab-9f34-85cc578c8cd5 None admin] PATH_INFO: `/v3/users` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:07.868206 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d5d2bc3-e049-4fab-9f34-85cc578c8cd5 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 10:52:07.868504 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d5d2bc3-e049-4fab-9f34-85cc578c8cd5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:52:07.870094 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d5d2bc3-e049-4fab-9f34-85cc578c8cd5 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:52:07.895879 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 266/530] 10.222.0.40 () {56 vars in 1139 bytes} [Fri Aug 26 10:52:07 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 10:52:07.895879 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d5d2bc3-e049-4fab-9f34-85cc578c8cd5 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:52:07.905947 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d21e5e-26ab-4256-80c5-83c72ea5c13f None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:52:07.941750 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0c63f4-4485-41ce-b078-7d1b6be5e324 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:52:07.942394 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0c63f4-4485-41ce-b078-7d1b6be5e324 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:07.942679 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0c63f4-4485-41ce-b078-7d1b6be5e324 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:07.942954 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0c63f4-4485-41ce-b078-7d1b6be5e324 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:07.943684 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e0c63f4-4485-41ce-b078-7d1b6be5e324 None admin] RBAC: Adding query filter params (role.id=3bda770fcd5c46cdbfabcd7404a25e66, scope.project.id=0e30ccf0c1204727b786876c141dbcd2, user.id=1f4b4a662db049ecacc865ecff10bdb0) {{(pid=81423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 10:52:07.944004 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e0c63f4-4485-41ce-b078-7d1b6be5e324 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:52:07.945437 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e0c63f4-4485-41ce-b078-7d1b6be5e324 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:52:07.950985 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 265/531] 10.222.0.40 () {56 vars in 1374 bytes} [Fri Aug 26 10:52:07 2022] GET /identity/v3/role_assignments?user.id=1f4b4a662db049ecacc865ecff10bdb0&scope.project.id=0e30ccf0c1204727b786876c141dbcd2&role.id=3bda770fcd5c46cdbfabcd7404a25e66 => generated 259 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 10:52:07.951768 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0c63f4-4485-41ce-b078-7d1b6be5e324 None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:52:08.025692 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d5d2bc3-e049-4fab-9f34-85cc578c8cd5 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:52:08.075343 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c289ef-4df1-4cdd-a1b1-e09368f09845 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:52:08.075913 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c289ef-4df1-4cdd-a1b1-e09368f09845 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:08.076193 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c289ef-4df1-4cdd-a1b1-e09368f09845 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:08.076464 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c289ef-4df1-4cdd-a1b1-e09368f09845 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:08.085841 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c289ef-4df1-4cdd-a1b1-e09368f09845 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:52:08.087985 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c289ef-4df1-4cdd-a1b1-e09368f09845 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:52:08.090876 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:52:08.099755 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0c63f4-4485-41ce-b078-7d1b6be5e324 None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:52:08.104358 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application [None req-38c289ef-4df1-4cdd-a1b1-e09368f09845 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 10:52:08.104358 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 10:52:08.104358 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 10:52:08.104358 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 10:52:08.104358 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 10:52:08.104358 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 10:52:08.104358 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 10:52:08.104358 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 10:52:08.104358 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 10:52:08.104358 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 10:52:08.104358 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 10:52:08.104358 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 10:52:08.104358 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 26 10:52:08.104358 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 26 10:52:08.104358 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 26 10:52:08.105244 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 26 10:52:08.105244 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 10:52:08.105244 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 10:52:08.105244 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 10:52:08.105244 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 10:52:08.105244 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 10:52:08.105244 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 10:52:08.105244 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 10:52:08.105244 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application with Lock( Aug 26 10:52:08.105244 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 10:52:08.105244 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._enter() Aug 26 10:52:08.105244 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 10:52:08.105244 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 10:52:08.105244 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 10:52:08.105244 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.creator() Aug 26 10:52:08.105244 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 10:52:08.105244 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application created_value = creator( Aug 26 10:52:08.106110 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 26 10:52:08.106110 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 26 10:52:08.106110 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 26 10:52:08.106110 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 26 10:52:08.106110 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 26 10:52:08.106110 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 26 10:52:08.106110 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 10:52:08.106110 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Aug 26 10:52:08.110338 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 267/532] 10.222.0.40 () {56 vars in 1111 bytes} [Fri Aug 26 10:52:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 10:52:08.111790 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c289ef-4df1-4cdd-a1b1-e09368f09845 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:52:08.145384 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c39516-c2b5-4b20-b479-96d7b2f18d63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:52:08.146040 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c39516-c2b5-4b20-b479-96d7b2f18d63 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:08.146329 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c39516-c2b5-4b20-b479-96d7b2f18d63 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:08.146604 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c39516-c2b5-4b20-b479-96d7b2f18d63 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:08.147332 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c39516-c2b5-4b20-b479-96d7b2f18d63 None admin] RBAC: Adding query filter params (role.id=3bda770fcd5c46cdbfabcd7404a25e66, scope.project.id=0e30ccf0c1204727b786876c141dbcd2, user.id=8631d7ecfd2849daa8694aecccd22a05) {{(pid=81423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 10:52:08.147638 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c39516-c2b5-4b20-b479-96d7b2f18d63 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:52:08.149061 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c39516-c2b5-4b20-b479-96d7b2f18d63 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:52:08.162180 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 266/533] 10.222.0.40 () {56 vars in 1374 bytes} [Fri Aug 26 10:52:07 2022] GET /identity/v3/role_assignments?user.id=8631d7ecfd2849daa8694aecccd22a05&scope.project.id=0e30ccf0c1204727b786876c141dbcd2&role.id=3bda770fcd5c46cdbfabcd7404a25e66 => generated 259 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 10:52:08.163214 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c39516-c2b5-4b20-b479-96d7b2f18d63 None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:52:08.178377 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:52:08.236894 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c289ef-4df1-4cdd-a1b1-e09368f09845 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:52:08.298178 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656341c9-45a4-4097-94b2-c12297e4db38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:52:08.299396 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656341c9-45a4-4097-94b2-c12297e4db38 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:08.299674 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656341c9-45a4-4097-94b2-c12297e4db38 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:08.299939 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656341c9-45a4-4097-94b2-c12297e4db38 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:08.381262 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c39516-c2b5-4b20-b479-96d7b2f18d63 None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:52:08.429386 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69527c80-be2f-4d58-bbf7-90784ac95d85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:52:08.430173 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69527c80-be2f-4d58-bbf7-90784ac95d85 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:08.430457 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69527c80-be2f-4d58-bbf7-90784ac95d85 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:08.430728 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69527c80-be2f-4d58-bbf7-90784ac95d85 None admin] PATH_INFO: `/v3/projects` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:08.431926 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69527c80-be2f-4d58-bbf7-90784ac95d85 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 10:52:08.432234 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69527c80-be2f-4d58-bbf7-90784ac95d85 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:52:08.433332 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-656341c9-45a4-4097-94b2-c12297e4db38 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:52:08.433960 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69527c80-be2f-4d58-bbf7-90784ac95d85 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:52:08.434832 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-656341c9-45a4-4097-94b2-c12297e4db38 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:52:08.449352 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 267/534] 10.222.0.40 () {56 vars in 1148 bytes} [Fri Aug 26 10:52:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 10:52:08.460272 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69527c80-be2f-4d58-bbf7-90784ac95d85 None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:52:08.502415 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:52:08.505827 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:52:08.557252 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 268/535] 10.222.0.40 () {58 vars in 1282 bytes} [Fri Aug 26 10:52:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 446 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 26 10:52:08.565268 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656341c9-45a4-4097-94b2-c12297e4db38 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:52:08.646565 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69527c80-be2f-4d58-bbf7-90784ac95d85 None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:52:08.683302 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5b7173-3d2c-4c74-ab8f-15cad2d2f62b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:52:08.683918 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5b7173-3d2c-4c74-ab8f-15cad2d2f62b None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:08.684197 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5b7173-3d2c-4c74-ab8f-15cad2d2f62b None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:08.684468 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5b7173-3d2c-4c74-ab8f-15cad2d2f62b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:08.685196 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d5b7173-3d2c-4c74-ab8f-15cad2d2f62b None admin] RBAC: Adding query filter params (role.id=3bda770fcd5c46cdbfabcd7404a25e66, scope.project.id=0e30ccf0c1204727b786876c141dbcd2, user.id=c277338a7f814cc6b01c5419775031bf) {{(pid=81423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 10:52:08.685502 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d5b7173-3d2c-4c74-ab8f-15cad2d2f62b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:52:08.686955 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d5b7173-3d2c-4c74-ab8f-15cad2d2f62b None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:52:08.698120 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 268/536] 10.222.0.40 () {56 vars in 1374 bytes} [Fri Aug 26 10:52:08 2022] GET /identity/v3/role_assignments?user.id=c277338a7f814cc6b01c5419775031bf&scope.project.id=0e30ccf0c1204727b786876c141dbcd2&role.id=3bda770fcd5c46cdbfabcd7404a25e66 => generated 259 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 10:52:08.810114 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656341c9-45a4-4097-94b2-c12297e4db38 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:52:08.849906 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35244177-ab88-49f9-b80a-fd65251cff93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:52:08.850570 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35244177-ab88-49f9-b80a-fd65251cff93 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:08.850850 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35244177-ab88-49f9-b80a-fd65251cff93 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:08.851126 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35244177-ab88-49f9-b80a-fd65251cff93 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:08.864335 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35244177-ab88-49f9-b80a-fd65251cff93 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:52:08.865813 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35244177-ab88-49f9-b80a-fd65251cff93 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:52:08.879869 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:52:08.890796 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application [None req-35244177-ab88-49f9-b80a-fd65251cff93 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 10:52:08.890796 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 10:52:08.890796 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 10:52:08.890796 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 10:52:08.890796 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 10:52:08.890796 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 10:52:08.890796 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 10:52:08.890796 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 10:52:08.890796 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 10:52:08.890796 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 10:52:08.890796 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 10:52:08.890796 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 10:52:08.890796 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 10:52:08.890796 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 10:52:08.890796 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 10:52:08.890796 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 10:52:08.891700 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 10:52:08.891700 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 10:52:08.891700 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 10:52:08.891700 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 10:52:08.891700 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 10:52:08.891700 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 10:52:08.891700 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 10:52:08.891700 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application with Lock( Aug 26 10:52:08.891700 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 10:52:08.891700 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._enter() Aug 26 10:52:08.891700 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 10:52:08.891700 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 10:52:08.891700 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 10:52:08.891700 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.creator() Aug 26 10:52:08.891700 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 10:52:08.891700 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application created_value = creator( Aug 26 10:52:08.891700 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 10:52:08.893089 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 10:52:08.893089 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 10:52:08.893089 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Aug 26 10:52:08.893089 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 269/537] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Aug 26 10:52:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 328 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 26 10:52:08.903113 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5b7173-3d2c-4c74-ab8f-15cad2d2f62b None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:52:09.018657 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:52:09.028517 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5b7173-3d2c-4c74-ab8f-15cad2d2f62b None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:52:09.055513 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cd0c52-840f-44ab-aa60-bc6863ae6ad4 None None] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:09.055848 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cd0c52-840f-44ab-aa60-bc6863ae6ad4 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:09.056073 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cd0c52-840f-44ab-aa60-bc6863ae6ad4 None None] PATH_INFO: `/` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:09.057690 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 270/538] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 26 10:52:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 10:52:09.063121 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07dd035a-9c4a-4bbc-83fc-68928f78aa2f None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:09.063406 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07dd035a-9c4a-4bbc-83fc-68928f78aa2f None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:09.063684 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07dd035a-9c4a-4bbc-83fc-68928f78aa2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:09.075459 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe6bd98-d858-4cb9-9e42-e4c4965e268b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:52:09.076344 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe6bd98-d858-4cb9-9e42-e4c4965e268b None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:09.076811 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe6bd98-d858-4cb9-9e42-e4c4965e268b None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:09.077272 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe6bd98-d858-4cb9-9e42-e4c4965e268b None admin] PATH_INFO: `/v3/domains` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:09.078244 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe6bd98-d858-4cb9-9e42-e4c4965e268b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 10:52:09.078736 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe6bd98-d858-4cb9-9e42-e4c4965e268b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:52:09.081064 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe6bd98-d858-4cb9-9e42-e4c4965e268b None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:52:09.094274 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 269/539] 10.222.0.40 () {56 vars in 1109 bytes} [Fri Aug 26 10:52:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 10:52:09.101801 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe6bd98-d858-4cb9-9e42-e4c4965e268b None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:52:09.105641 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-07dd035a-9c4a-4bbc-83fc-68928f78aa2f None None] MFA Rules not processed for user `ca155a48e2024c32af3fd2e19e5fac11`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 10:52:09.107932 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-07dd035a-9c4a-4bbc-83fc-68928f78aa2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 10:52:09.217626 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 271/540] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 26 10:52:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 10:52:09.266378 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe6bd98-d858-4cb9-9e42-e4c4965e268b None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:52:09.303866 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429dd5c4-2531-4a0e-bd07-d5c2332acd3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:52:09.304510 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429dd5c4-2531-4a0e-bd07-d5c2332acd3b None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:09.304790 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429dd5c4-2531-4a0e-bd07-d5c2332acd3b None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:09.305063 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429dd5c4-2531-4a0e-bd07-d5c2332acd3b None admin] PATH_INFO: `/v3/projects/service` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:09.312034 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-429dd5c4-2531-4a0e-bd07-d5c2332acd3b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:52:09.313482 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-429dd5c4-2531-4a0e-bd07-d5c2332acd3b None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:52:09.325885 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application [None req-429dd5c4-2531-4a0e-bd07-d5c2332acd3b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 10:52:09.325885 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 10:52:09.325885 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 10:52:09.325885 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 10:52:09.325885 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 10:52:09.325885 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 10:52:09.325885 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 10:52:09.325885 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 10:52:09.325885 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 10:52:09.325885 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 10:52:09.325885 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 10:52:09.325885 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 10:52:09.325885 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 26 10:52:09.325885 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 26 10:52:09.325885 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 26 10:52:09.326747 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 26 10:52:09.326747 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 10:52:09.326747 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 10:52:09.326747 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 10:52:09.326747 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 10:52:09.326747 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 10:52:09.326747 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 10:52:09.326747 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 10:52:09.326747 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application with Lock( Aug 26 10:52:09.326747 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 10:52:09.326747 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._enter() Aug 26 10:52:09.326747 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 10:52:09.326747 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 10:52:09.326747 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 10:52:09.326747 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.creator() Aug 26 10:52:09.326747 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 10:52:09.326747 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application created_value = creator( Aug 26 10:52:09.327579 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 26 10:52:09.327579 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 26 10:52:09.327579 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 26 10:52:09.327579 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 26 10:52:09.327579 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 26 10:52:09.327579 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 26 10:52:09.327579 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 10:52:09.327579 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Aug 26 10:52:09.327579 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 270/541] 10.222.0.40 () {56 vars in 1111 bytes} [Fri Aug 26 10:52:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 10:52:09.333673 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07dd035a-9c4a-4bbc-83fc-68928f78aa2f None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:52:09.478388 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07dd035a-9c4a-4bbc-83fc-68928f78aa2f None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:52:09.493492 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429dd5c4-2531-4a0e-bd07-d5c2332acd3b None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:52:09.499079 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:52:09.521512 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:52:09.528370 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29062f31-b5cc-49ca-8f11-c19c6187c1a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:52:09.528981 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29062f31-b5cc-49ca-8f11-c19c6187c1a7 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:09.529254 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29062f31-b5cc-49ca-8f11-c19c6187c1a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:09.529534 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29062f31-b5cc-49ca-8f11-c19c6187c1a7 None admin] PATH_INFO: `/v3/projects` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:09.530234 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29062f31-b5cc-49ca-8f11-c19c6187c1a7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 10:52:09.530539 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29062f31-b5cc-49ca-8f11-c19c6187c1a7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:52:09.532771 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29062f31-b5cc-49ca-8f11-c19c6187c1a7 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:52:09.571974 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 272/542] 10.222.0.40 () {56 vars in 1148 bytes} [Fri Aug 26 10:52:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 10:52:09.579126 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29062f31-b5cc-49ca-8f11-c19c6187c1a7 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:52:09.683675 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29062f31-b5cc-49ca-8f11-c19c6187c1a7 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:52:09.684789 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429dd5c4-2531-4a0e-bd07-d5c2332acd3b None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:52:09.718847 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf3caa3-bc15-4195-ac88-a4ae855dc044 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:52:09.719456 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf3caa3-bc15-4195-ac88-a4ae855dc044 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:09.719729 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf3caa3-bc15-4195-ac88-a4ae855dc044 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:09.720103 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf3caa3-bc15-4195-ac88-a4ae855dc044 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:09.720910 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf3caa3-bc15-4195-ac88-a4ae855dc044 None admin] RBAC: Adding query filter params (role.id=3bda770fcd5c46cdbfabcd7404a25e66, scope.project.id=0e30ccf0c1204727b786876c141dbcd2, user.id=91da2acd2534423091537e1ba068d2b7) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 10:52:09.721214 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf3caa3-bc15-4195-ac88-a4ae855dc044 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:52:09.722704 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf3caa3-bc15-4195-ac88-a4ae855dc044 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:52:09.728944 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 273/543] 10.222.0.40 () {56 vars in 1374 bytes} [Fri Aug 26 10:52:09 2022] GET /identity/v3/role_assignments?user.id=91da2acd2534423091537e1ba068d2b7&scope.project.id=0e30ccf0c1204727b786876c141dbcd2&role.id=3bda770fcd5c46cdbfabcd7404a25e66 => generated 259 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 10:52:09.772838 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce1426b-6a68-49c5-8441-85c93ab118b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:52:09.773496 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce1426b-6a68-49c5-8441-85c93ab118b0 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:09.773772 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce1426b-6a68-49c5-8441-85c93ab118b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:09.774067 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce1426b-6a68-49c5-8441-85c93ab118b0 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:09.799094 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce1426b-6a68-49c5-8441-85c93ab118b0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 10:52:09.804020 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application [None req-5ce1426b-6a68-49c5-8441-85c93ab118b0 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 26 10:52:09.804020 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 10:52:09.804020 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 10:52:09.804020 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 10:52:09.804020 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 10:52:09.804020 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 10:52:09.804020 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 10:52:09.804020 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 10:52:09.804020 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 10:52:09.804020 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 10:52:09.804020 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 10:52:09.804020 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 10:52:09.804020 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 10:52:09.804020 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 10:52:09.804020 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 10:52:09.804020 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 10:52:09.804948 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 10:52:09.804948 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 10:52:09.804948 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 10:52:09.804948 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 10:52:09.804948 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 10:52:09.804948 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 10:52:09.804948 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 10:52:09.804948 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 10:52:09.804948 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 10:52:09.804948 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 10:52:09.804948 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 10:52:09.804948 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application with Lock( Aug 26 10:52:09.804948 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 10:52:09.804948 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._enter() Aug 26 10:52:09.804948 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 10:52:09.804948 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 10:52:09.805772 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 10:52:09.805772 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.creator() Aug 26 10:52:09.805772 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 10:52:09.805772 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application created_value = creator( Aug 26 10:52:09.805772 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 10:52:09.805772 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 10:52:09.805772 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 10:52:09.805772 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 10:52:09.805772 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 10:52:09.805772 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 10:52:09.805772 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 26 10:52:09.805772 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Aug 26 10:52:09.805772 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 271/544] 10.222.0.40 () {56 vars in 1102 bytes} [Fri Aug 26 10:52:09 2022] GET /identity/v3/roles/service => generated 85 bytes in 312 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 10:52:09.811460 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf3caa3-bc15-4195-ac88-a4ae855dc044 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:52:09.890445 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:52:09.935126 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf3caa3-bc15-4195-ac88-a4ae855dc044 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:52:09.970081 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245a3917-6416-400e-bdc5-ffda8dcebd40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:52:09.970846 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245a3917-6416-400e-bdc5-ffda8dcebd40 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:09.971223 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245a3917-6416-400e-bdc5-ffda8dcebd40 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:09.971491 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245a3917-6416-400e-bdc5-ffda8dcebd40 None admin] PATH_INFO: `/v3/roles` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:09.972158 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-245a3917-6416-400e-bdc5-ffda8dcebd40 None admin] RBAC: Adding query filter params (name=service) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 10:52:09.972454 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-245a3917-6416-400e-bdc5-ffda8dcebd40 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:52:09.973824 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-245a3917-6416-400e-bdc5-ffda8dcebd40 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:52:09.982706 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 274/545] 10.222.0.40 () {56 vars in 1103 bytes} [Fri Aug 26 10:52:09 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 10:52:09.988167 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce1426b-6a68-49c5-8441-85c93ab118b0 None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:52:10.111340 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:52:10.158425 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce1426b-6a68-49c5-8441-85c93ab118b0 None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:52:10.205531 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800422ab-ce07-47ab-9aaa-a72f74ff6a8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:52:10.206195 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800422ab-ce07-47ab-9aaa-a72f74ff6a8b None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:10.206476 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800422ab-ce07-47ab-9aaa-a72f74ff6a8b None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:10.206746 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800422ab-ce07-47ab-9aaa-a72f74ff6a8b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:10.220357 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-800422ab-ce07-47ab-9aaa-a72f74ff6a8b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:52:10.221874 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-800422ab-ce07-47ab-9aaa-a72f74ff6a8b None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:52:10.242738 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application [None req-800422ab-ce07-47ab-9aaa-a72f74ff6a8b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 10:52:10.242738 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 10:52:10.242738 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 10:52:10.242738 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 10:52:10.242738 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 10:52:10.242738 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 10:52:10.242738 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 10:52:10.242738 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 10:52:10.242738 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 10:52:10.242738 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 10:52:10.242738 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 10:52:10.242738 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 10:52:10.242738 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 10:52:10.242738 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 10:52:10.242738 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 10:52:10.242738 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 10:52:10.243668 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 10:52:10.243668 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 10:52:10.243668 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 10:52:10.243668 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 10:52:10.243668 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 10:52:10.243668 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 10:52:10.243668 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 10:52:10.243668 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application with Lock( Aug 26 10:52:10.243668 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 10:52:10.243668 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._enter() Aug 26 10:52:10.243668 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 10:52:10.243668 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 10:52:10.243668 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 10:52:10.243668 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.creator() Aug 26 10:52:10.243668 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 10:52:10.243668 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application created_value = creator( Aug 26 10:52:10.243668 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 10:52:10.244500 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 10:52:10.244500 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 10:52:10.244500 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Aug 26 10:52:10.244500 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 272/546] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Aug 26 10:52:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 10:52:10.251044 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245a3917-6416-400e-bdc5-ffda8dcebd40 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:52:10.342547 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0caaa750-12ca-4b2f-a638-5819af5aa270 None None] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:10.343171 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0caaa750-12ca-4b2f-a638-5819af5aa270 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:10.343644 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0caaa750-12ca-4b2f-a638-5819af5aa270 None None] PATH_INFO: `/` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:10.345537 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 273/547] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 26 10:52:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 26 10:52:10.351064 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fd8f4e-02fd-4ea3-8e9b-5f5e436a977d None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:10.351594 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fd8f4e-02fd-4ea3-8e9b-5f5e436a977d None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:10.352069 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fd8f4e-02fd-4ea3-8e9b-5f5e436a977d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:10.386088 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245a3917-6416-400e-bdc5-ffda8dcebd40 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:52:10.391732 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-f1fd8f4e-02fd-4ea3-8e9b-5f5e436a977d None None] MFA Rules not processed for user `ca155a48e2024c32af3fd2e19e5fac11`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 10:52:10.393329 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-f1fd8f4e-02fd-4ea3-8e9b-5f5e436a977d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 10:52:10.397703 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:52:10.415806 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9fe27a-d2cf-4770-9b7d-ed7efa42a1d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:52:10.416468 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9fe27a-d2cf-4770-9b7d-ed7efa42a1d3 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:10.416750 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9fe27a-d2cf-4770-9b7d-ed7efa42a1d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:10.417025 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9fe27a-d2cf-4770-9b7d-ed7efa42a1d3 None admin] PATH_INFO: `/v3/domains` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:10.417655 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad9fe27a-d2cf-4770-9b7d-ed7efa42a1d3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 10:52:10.418016 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad9fe27a-d2cf-4770-9b7d-ed7efa42a1d3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:52:10.419398 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad9fe27a-d2cf-4770-9b7d-ed7efa42a1d3 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:52:10.432739 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 275/548] 10.222.0.40 () {56 vars in 1109 bytes} [Fri Aug 26 10:52:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 10:52:10.437775 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9fe27a-d2cf-4770-9b7d-ed7efa42a1d3 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:52:10.542040 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9fe27a-d2cf-4770-9b7d-ed7efa42a1d3 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:52:10.546558 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 274/549] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 26 10:52:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 26 10:52:10.548000 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8338fdf6-38f4-478b-a273-2f4598752e60 None None] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:10.548873 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8338fdf6-38f4-478b-a273-2f4598752e60 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:10.549273 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8338fdf6-38f4-478b-a273-2f4598752e60 None None] PATH_INFO: `/` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:10.552016 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 275/550] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 26 10:52:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 10:52:10.556294 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6cdc22d-639b-42a8-a210-a52fcd73d519 None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:10.556577 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6cdc22d-639b-42a8-a210-a52fcd73d519 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:10.556848 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6cdc22d-639b-42a8-a210-a52fcd73d519 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:10.575884 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-a6cdc22d-639b-42a8-a210-a52fcd73d519 None None] MFA Rules not processed for user `ca155a48e2024c32af3fd2e19e5fac11`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 10:52:10.577000 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-a6cdc22d-639b-42a8-a210-a52fcd73d519 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 10:52:10.583247 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb31df56-d678-4a1a-8ae1-224d3a0aab73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:52:10.583946 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb31df56-d678-4a1a-8ae1-224d3a0aab73 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:10.584219 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb31df56-d678-4a1a-8ae1-224d3a0aab73 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:10.584490 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb31df56-d678-4a1a-8ae1-224d3a0aab73 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:10.600580 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb31df56-d678-4a1a-8ae1-224d3a0aab73 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:52:10.602050 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb31df56-d678-4a1a-8ae1-224d3a0aab73 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:52:10.621855 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application [None req-eb31df56-d678-4a1a-8ae1-224d3a0aab73 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 26 10:52:10.621855 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 10:52:10.621855 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 10:52:10.621855 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 10:52:10.621855 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 10:52:10.621855 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 10:52:10.621855 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 10:52:10.621855 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 10:52:10.621855 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 10:52:10.621855 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 10:52:10.621855 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 10:52:10.621855 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 10:52:10.621855 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 26 10:52:10.621855 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 26 10:52:10.621855 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 26 10:52:10.621855 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 26 10:52:10.623383 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 10:52:10.623383 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 10:52:10.623383 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 26 10:52:10.623383 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 10:52:10.623383 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 26 10:52:10.623383 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 10:52:10.623383 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 10:52:10.623383 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 10:52:10.623383 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 10:52:10.623383 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 10:52:10.623383 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 10:52:10.623383 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application with Lock( Aug 26 10:52:10.623383 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 10:52:10.623383 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._enter() Aug 26 10:52:10.623383 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 10:52:10.623383 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 10:52:10.623383 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 10:52:10.623383 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.creator() Aug 26 10:52:10.624373 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 10:52:10.624373 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application created_value = creator( Aug 26 10:52:10.624373 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 26 10:52:10.624373 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 26 10:52:10.624373 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 26 10:52:10.624373 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 26 10:52:10.624373 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 26 10:52:10.624373 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 26 10:52:10.624373 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 26 10:52:10.624373 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Aug 26 10:52:10.628830 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 276/551] 10.222.0.40 () {56 vars in 1096 bytes} [Fri Aug 26 10:52:10 2022] GET /identity/v3/users/swift => generated 83 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 26 10:52:10.629697 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb31df56-d678-4a1a-8ae1-224d3a0aab73 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:52:10.634525 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:52:10.677113 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 276/552] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 26 10:52:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 26 10:52:10.695579 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:52:10.781660 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb31df56-d678-4a1a-8ae1-224d3a0aab73 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:52:10.826906 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e827347-7a5d-49b3-aaa9-6171f520a64e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:52:10.827442 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e827347-7a5d-49b3-aaa9-6171f520a64e None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:10.827722 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e827347-7a5d-49b3-aaa9-6171f520a64e None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:10.827994 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e827347-7a5d-49b3-aaa9-6171f520a64e None admin] PATH_INFO: `/v3/users` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:10.828727 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e827347-7a5d-49b3-aaa9-6171f520a64e None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 10:52:10.829204 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e827347-7a5d-49b3-aaa9-6171f520a64e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:52:10.831257 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e827347-7a5d-49b3-aaa9-6171f520a64e None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:52:10.848190 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6cdc22d-639b-42a8-a210-a52fcd73d519 None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:52:10.860382 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 277/553] 10.222.0.40 () {56 vars in 1135 bytes} [Fri Aug 26 10:52:10 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 10:52:10.866218 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e827347-7a5d-49b3-aaa9-6171f520a64e None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:52:11.030364 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:52:11.035672 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e827347-7a5d-49b3-aaa9-6171f520a64e None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:52:11.069805 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6cdc22d-639b-42a8-a210-a52fcd73d519 None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:52:11.098498 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca16f8c8-3427-48ec-9950-d17b5b259487 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:52:11.099037 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca16f8c8-3427-48ec-9950-d17b5b259487 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:11.099316 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca16f8c8-3427-48ec-9950-d17b5b259487 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:11.099588 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca16f8c8-3427-48ec-9950-d17b5b259487 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:11.112144 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc43b2d-fcb1-4297-af59-ea56d79a982b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:52:11.112775 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc43b2d-fcb1-4297-af59-ea56d79a982b None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:11.113045 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc43b2d-fcb1-4297-af59-ea56d79a982b None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:11.113314 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc43b2d-fcb1-4297-af59-ea56d79a982b None admin] PATH_INFO: `/v3/roles/service` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:11.123474 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc43b2d-fcb1-4297-af59-ea56d79a982b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 10:52:11.129014 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application [None req-0dc43b2d-fcb1-4297-af59-ea56d79a982b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 26 10:52:11.129014 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 10:52:11.129014 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 10:52:11.129014 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 10:52:11.129014 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 10:52:11.129014 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 10:52:11.129014 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 10:52:11.129014 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 10:52:11.129014 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 10:52:11.129014 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 10:52:11.129014 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 10:52:11.129014 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 10:52:11.129014 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 10:52:11.129014 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 10:52:11.129014 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 10:52:11.129014 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 10:52:11.130084 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 10:52:11.130084 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 10:52:11.130084 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 10:52:11.130084 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 10:52:11.130084 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 10:52:11.130084 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 10:52:11.130084 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 10:52:11.130084 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 10:52:11.130084 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 10:52:11.130084 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 10:52:11.130084 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 10:52:11.130084 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application with Lock( Aug 26 10:52:11.130084 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 10:52:11.130084 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._enter() Aug 26 10:52:11.130084 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 10:52:11.130084 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 10:52:11.130904 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 10:52:11.130904 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.creator() Aug 26 10:52:11.130904 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 10:52:11.130904 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application created_value = creator( Aug 26 10:52:11.130904 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 10:52:11.130904 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 10:52:11.130904 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 10:52:11.130904 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 10:52:11.130904 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 10:52:11.130904 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 10:52:11.130904 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 26 10:52:11.130904 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Aug 26 10:52:11.134878 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 277/554] 10.222.0.40 () {56 vars in 1102 bytes} [Fri Aug 26 10:52:10 2022] GET /identity/v3/roles/service => generated 85 bytes in 287 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 26 10:52:11.136100 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a949630-1afd-43eb-b324-d03d6f18a7c4 None None] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:11.136386 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a949630-1afd-43eb-b324-d03d6f18a7c4 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:11.136661 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a949630-1afd-43eb-b324-d03d6f18a7c4 None None] PATH_INFO: `/` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:11.137894 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 278/555] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 26 10:52:11 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 10:52:11.138600 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a949630-1afd-43eb-b324-d03d6f18a7c4 None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:52:11.176216 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:52:11.186271 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca16f8c8-3427-48ec-9950-d17b5b259487 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:52:11.187862 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca16f8c8-3427-48ec-9950-d17b5b259487 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:52:11.243587 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a949630-1afd-43eb-b324-d03d6f18a7c4 None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:52:11.279727 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6ee1da-3240-4308-90b9-71046d92aa11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:52:11.280360 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6ee1da-3240-4308-90b9-71046d92aa11 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:11.280639 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6ee1da-3240-4308-90b9-71046d92aa11 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:11.280909 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6ee1da-3240-4308-90b9-71046d92aa11 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:11.291107 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:52:11.292043 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be6ee1da-3240-4308-90b9-71046d92aa11 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 10:52:11.299141 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application [None req-be6ee1da-3240-4308-90b9-71046d92aa11 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 26 10:52:11.299141 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 10:52:11.299141 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 10:52:11.299141 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 10:52:11.299141 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 10:52:11.299141 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 10:52:11.299141 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 10:52:11.299141 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 10:52:11.299141 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 10:52:11.299141 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 10:52:11.299141 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 10:52:11.299141 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 10:52:11.299141 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 10:52:11.299141 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 10:52:11.299141 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 10:52:11.299141 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 10:52:11.300146 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 10:52:11.300146 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 10:52:11.300146 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 10:52:11.300146 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 10:52:11.300146 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 10:52:11.300146 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 10:52:11.300146 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 10:52:11.300146 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 10:52:11.300146 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 10:52:11.300146 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 10:52:11.300146 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 10:52:11.300146 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application with Lock( Aug 26 10:52:11.300146 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 10:52:11.300146 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._enter() Aug 26 10:52:11.300146 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 10:52:11.300146 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 10:52:11.300989 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 10:52:11.300989 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.creator() Aug 26 10:52:11.300989 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 10:52:11.300989 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application created_value = creator( Aug 26 10:52:11.300989 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 10:52:11.300989 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 10:52:11.300989 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 10:52:11.300989 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 10:52:11.300989 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 10:52:11.300989 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 10:52:11.300989 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 26 10:52:11.300989 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Aug 26 10:52:11.301614 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 278/556] 10.222.0.40 () {58 vars in 1282 bytes} [Fri Aug 26 10:52:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 435 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 26 10:52:11.302331 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca16f8c8-3427-48ec-9950-d17b5b259487 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:52:11.309239 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 279/557] 10.222.0.40 () {56 vars in 1102 bytes} [Fri Aug 26 10:52:11 2022] GET /identity/v3/roles/service => generated 85 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 10:52:11.310552 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8715099-be40-4b1d-b9b7-400b43ff6a93 None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:11.310837 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8715099-be40-4b1d-b9b7-400b43ff6a93 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:11.311111 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8715099-be40-4b1d-b9b7-400b43ff6a93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:11.331821 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-e8715099-be40-4b1d-b9b7-400b43ff6a93 None None] MFA Rules not processed for user `ca155a48e2024c32af3fd2e19e5fac11`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 10:52:11.332839 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-e8715099-be40-4b1d-b9b7-400b43ff6a93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 10:52:11.407023 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 280/558] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 26 10:52:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 10:52:11.408491 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8715099-be40-4b1d-b9b7-400b43ff6a93 None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:52:11.417438 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca16f8c8-3427-48ec-9950-d17b5b259487 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:52:11.457593 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ddf8016-7013-4b95-8fc8-c93d11632459 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:52:11.458022 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:52:11.458789 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddf8016-7013-4b95-8fc8-c93d11632459 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:11.458789 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddf8016-7013-4b95-8fc8-c93d11632459 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:11.459274 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddf8016-7013-4b95-8fc8-c93d11632459 None admin] PATH_INFO: `/v3/roles` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:11.460087 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ddf8016-7013-4b95-8fc8-c93d11632459 None admin] RBAC: Adding query filter params (name=service) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 10:52:11.460655 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ddf8016-7013-4b95-8fc8-c93d11632459 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:52:11.462164 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ddf8016-7013-4b95-8fc8-c93d11632459 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:52:11.471900 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 279/559] 10.222.0.40 () {56 vars in 1103 bytes} [Fri Aug 26 10:52:11 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 10:52:11.472956 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ddf8016-7013-4b95-8fc8-c93d11632459 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:52:11.535255 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:52:11.570837 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8715099-be40-4b1d-b9b7-400b43ff6a93 None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:52:11.585296 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ddf8016-7013-4b95-8fc8-c93d11632459 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:52:11.619865 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b38552-597b-42ec-be13-b6333d559401 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:52:11.620799 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51880d2a-a58b-474b-bc21-d4306b51193f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:52:11.621018 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b38552-597b-42ec-be13-b6333d559401 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:11.621442 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b38552-597b-42ec-be13-b6333d559401 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:11.621694 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51880d2a-a58b-474b-bc21-d4306b51193f None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:11.622051 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b38552-597b-42ec-be13-b6333d559401 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:11.622180 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51880d2a-a58b-474b-bc21-d4306b51193f None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:11.622521 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51880d2a-a58b-474b-bc21-d4306b51193f None admin] PATH_INFO: `/v3/roles` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:11.623274 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51880d2a-a58b-474b-bc21-d4306b51193f None admin] RBAC: Adding query filter params (name=service) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 10:52:11.623655 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51880d2a-a58b-474b-bc21-d4306b51193f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:52:11.625087 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51880d2a-a58b-474b-bc21-d4306b51193f None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:52:11.641055 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b38552-597b-42ec-be13-b6333d559401 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:52:11.643122 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b38552-597b-42ec-be13-b6333d559401 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:52:11.643696 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 280/560] 10.222.0.40 () {56 vars in 1103 bytes} [Fri Aug 26 10:52:11 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 10:52:11.644669 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51880d2a-a58b-474b-bc21-d4306b51193f None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:52:11.667538 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application [None req-88b38552-597b-42ec-be13-b6333d559401 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 10:52:11.667538 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 10:52:11.667538 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 10:52:11.667538 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 10:52:11.667538 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 10:52:11.667538 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 10:52:11.667538 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 10:52:11.667538 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 10:52:11.667538 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 10:52:11.667538 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 10:52:11.667538 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 10:52:11.667538 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 10:52:11.667538 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 10:52:11.667538 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 10:52:11.667538 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 10:52:11.667538 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 10:52:11.668543 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 10:52:11.668543 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 10:52:11.668543 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 10:52:11.668543 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 10:52:11.668543 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 10:52:11.668543 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 10:52:11.668543 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 10:52:11.668543 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application with Lock( Aug 26 10:52:11.668543 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 10:52:11.668543 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._enter() Aug 26 10:52:11.668543 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 10:52:11.668543 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 10:52:11.668543 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 10:52:11.668543 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.creator() Aug 26 10:52:11.668543 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 10:52:11.668543 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application created_value = creator( Aug 26 10:52:11.668543 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 10:52:11.669520 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 10:52:11.669520 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 10:52:11.669520 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Aug 26 10:52:11.669520 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 281/561] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Aug 26 10:52:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 261 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 10:52:11.669520 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b38552-597b-42ec-be13-b6333d559401 None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:52:11.760209 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:52:11.769841 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51880d2a-a58b-474b-bc21-d4306b51193f None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:52:11.785111 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b38552-597b-42ec-be13-b6333d559401 None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:52:11.789895 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:52:11.810043 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc5bd4c-4957-4173-81fe-3b371bf1965c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:52:11.810677 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc5bd4c-4957-4173-81fe-3b371bf1965c None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:11.811191 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc5bd4c-4957-4173-81fe-3b371bf1965c None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:11.811191 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc5bd4c-4957-4173-81fe-3b371bf1965c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:11.822560 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc5bd4c-4957-4173-81fe-3b371bf1965c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:52:11.823181 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e03be63-7aca-448d-a05d-05bbd6ff0048 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:52:11.823870 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e03be63-7aca-448d-a05d-05bbd6ff0048 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:11.823993 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc5bd4c-4957-4173-81fe-3b371bf1965c None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:52:11.824255 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e03be63-7aca-448d-a05d-05bbd6ff0048 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:11.824639 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e03be63-7aca-448d-a05d-05bbd6ff0048 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:11.835587 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e03be63-7aca-448d-a05d-05bbd6ff0048 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:52:11.837017 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e03be63-7aca-448d-a05d-05bbd6ff0048 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:52:11.839981 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application [None req-5fc5bd4c-4957-4173-81fe-3b371bf1965c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 10:52:11.839981 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 10:52:11.839981 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 10:52:11.839981 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 10:52:11.839981 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 10:52:11.839981 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 10:52:11.839981 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 10:52:11.839981 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 10:52:11.839981 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 10:52:11.839981 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 10:52:11.839981 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 10:52:11.839981 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 10:52:11.839981 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 10:52:11.839981 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 10:52:11.839981 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 10:52:11.839981 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 10:52:11.840900 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 10:52:11.840900 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 10:52:11.840900 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 10:52:11.840900 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 10:52:11.840900 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 10:52:11.840900 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 10:52:11.840900 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 10:52:11.840900 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application with Lock( Aug 26 10:52:11.840900 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 10:52:11.840900 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._enter() Aug 26 10:52:11.840900 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 10:52:11.840900 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 10:52:11.840900 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 10:52:11.840900 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.creator() Aug 26 10:52:11.840900 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 10:52:11.840900 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application created_value = creator( Aug 26 10:52:11.840900 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 10:52:11.841769 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 10:52:11.841769 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 10:52:11.841769 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Aug 26 10:52:11.845783 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 281/562] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Aug 26 10:52:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 10:52:11.846654 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc5bd4c-4957-4173-81fe-3b371bf1965c None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:52:11.852956 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application [None req-1e03be63-7aca-448d-a05d-05bbd6ff0048 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 10:52:11.852956 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 10:52:11.852956 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 10:52:11.852956 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 10:52:11.852956 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 10:52:11.852956 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 10:52:11.852956 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 10:52:11.852956 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 10:52:11.852956 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 10:52:11.852956 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 10:52:11.852956 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 10:52:11.852956 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 10:52:11.852956 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 10:52:11.852956 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 10:52:11.852956 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 10:52:11.852956 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 10:52:11.854496 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 10:52:11.854496 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 10:52:11.854496 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 10:52:11.854496 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 10:52:11.854496 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 10:52:11.854496 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 10:52:11.854496 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 10:52:11.854496 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application with Lock( Aug 26 10:52:11.854496 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 10:52:11.854496 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._enter() Aug 26 10:52:11.854496 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 10:52:11.854496 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 10:52:11.854496 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 10:52:11.854496 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.creator() Aug 26 10:52:11.854496 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 10:52:11.854496 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application created_value = creator( Aug 26 10:52:11.854496 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 10:52:11.855362 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 10:52:11.855362 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 10:52:11.855362 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Aug 26 10:52:11.855362 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 282/563] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Aug 26 10:52:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 10:52:11.855362 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e03be63-7aca-448d-a05d-05bbd6ff0048 None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:52:11.961144 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc5bd4c-4957-4173-81fe-3b371bf1965c None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:52:11.971038 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e03be63-7aca-448d-a05d-05bbd6ff0048 None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:52:11.996258 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88bf95f0-bc20-46d0-88f1-faf4a51ad67a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:52:11.996876 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88bf95f0-bc20-46d0-88f1-faf4a51ad67a None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:11.997138 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88bf95f0-bc20-46d0-88f1-faf4a51ad67a None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:11.997411 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88bf95f0-bc20-46d0-88f1-faf4a51ad67a None admin] PATH_INFO: `/v3/domains` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:11.998091 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88bf95f0-bc20-46d0-88f1-faf4a51ad67a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 10:52:11.998394 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88bf95f0-bc20-46d0-88f1-faf4a51ad67a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:52:11.999753 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88bf95f0-bc20-46d0-88f1-faf4a51ad67a None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:52:12.003967 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:52:12.012402 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf215bc-8770-450a-900f-dbbe20e4aa7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:52:12.012576 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 282/564] 10.222.0.40 () {56 vars in 1109 bytes} [Fri Aug 26 10:52:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 10:52:12.017512 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf215bc-8770-450a-900f-dbbe20e4aa7b None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:12.017697 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88bf95f0-bc20-46d0-88f1-faf4a51ad67a None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:52:12.017813 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf215bc-8770-450a-900f-dbbe20e4aa7b None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:12.017813 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf215bc-8770-450a-900f-dbbe20e4aa7b None admin] PATH_INFO: `/v3/roles/service` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:12.028707 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf215bc-8770-450a-900f-dbbe20e4aa7b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 10:52:12.035308 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application [None req-3cf215bc-8770-450a-900f-dbbe20e4aa7b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 26 10:52:12.035308 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 10:52:12.035308 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 10:52:12.035308 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 10:52:12.035308 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 10:52:12.035308 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 10:52:12.035308 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 10:52:12.035308 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 10:52:12.035308 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 10:52:12.035308 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 10:52:12.035308 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 10:52:12.035308 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 10:52:12.035308 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 10:52:12.035308 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 10:52:12.035308 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 10:52:12.035308 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 10:52:12.036419 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 10:52:12.036419 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 10:52:12.036419 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 10:52:12.036419 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 10:52:12.036419 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 10:52:12.036419 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 10:52:12.036419 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 10:52:12.036419 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 10:52:12.036419 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 10:52:12.036419 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 10:52:12.036419 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 10:52:12.036419 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application with Lock( Aug 26 10:52:12.036419 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 10:52:12.036419 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._enter() Aug 26 10:52:12.036419 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 10:52:12.036419 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 10:52:12.037238 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 10:52:12.037238 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.creator() Aug 26 10:52:12.037238 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 10:52:12.037238 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application created_value = creator( Aug 26 10:52:12.037238 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 10:52:12.037238 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 10:52:12.037238 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 10:52:12.037238 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 10:52:12.037238 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 10:52:12.037238 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 10:52:12.037238 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 26 10:52:12.037238 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Aug 26 10:52:12.037238 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 283/565] 10.222.0.40 () {56 vars in 1102 bytes} [Fri Aug 26 10:52:11 2022] GET /identity/v3/roles/service => generated 85 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 10:52:12.037238 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf215bc-8770-450a-900f-dbbe20e4aa7b None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:52:12.054279 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:52:12.122948 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88bf95f0-bc20-46d0-88f1-faf4a51ad67a None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:52:12.136714 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf215bc-8770-450a-900f-dbbe20e4aa7b None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:52:12.157041 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59487337-e568-448d-80b2-2108cd765d60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:52:12.158172 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59487337-e568-448d-80b2-2108cd765d60 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:12.158447 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59487337-e568-448d-80b2-2108cd765d60 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:12.158716 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59487337-e568-448d-80b2-2108cd765d60 None admin] PATH_INFO: `/v3/domains` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:12.159353 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59487337-e568-448d-80b2-2108cd765d60 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 10:52:12.159656 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59487337-e568-448d-80b2-2108cd765d60 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:52:12.161022 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59487337-e568-448d-80b2-2108cd765d60 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:52:12.169754 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9b2434-35ee-448a-927f-2b9e55ed170d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:52:12.170374 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9b2434-35ee-448a-927f-2b9e55ed170d None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:12.170646 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9b2434-35ee-448a-927f-2b9e55ed170d None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:12.170912 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9b2434-35ee-448a-927f-2b9e55ed170d None admin] PATH_INFO: `/v3/domains` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:12.171542 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d9b2434-35ee-448a-927f-2b9e55ed170d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 10:52:12.171837 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d9b2434-35ee-448a-927f-2b9e55ed170d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:52:12.176948 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 283/566] 10.222.0.40 () {56 vars in 1109 bytes} [Fri Aug 26 10:52:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 10:52:12.176948 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59487337-e568-448d-80b2-2108cd765d60 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:52:12.177165 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d9b2434-35ee-448a-927f-2b9e55ed170d None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:52:12.187821 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 284/567] 10.222.0.40 () {56 vars in 1109 bytes} [Fri Aug 26 10:52:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 10:52:12.189051 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e3a08b-e584-4702-94e1-d16e6a6eeed2 None None] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:12.189373 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e3a08b-e584-4702-94e1-d16e6a6eeed2 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:12.189670 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e3a08b-e584-4702-94e1-d16e6a6eeed2 None None] PATH_INFO: `/` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:12.190918 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 285/568] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 26 10:52:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 10:52:12.191600 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e3a08b-e584-4702-94e1-d16e6a6eeed2 None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:52:12.243655 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:52:12.269138 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:52:12.278924 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59487337-e568-448d-80b2-2108cd765d60 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:52:12.294558 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e3a08b-e584-4702-94e1-d16e6a6eeed2 None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:52:12.311844 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc218a41-b40a-4249-a020-d8a0f2edb516 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:52:12.312463 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc218a41-b40a-4249-a020-d8a0f2edb516 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:12.312739 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc218a41-b40a-4249-a020-d8a0f2edb516 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:12.313009 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc218a41-b40a-4249-a020-d8a0f2edb516 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:12.320213 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc218a41-b40a-4249-a020-d8a0f2edb516 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:52:12.321641 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc218a41-b40a-4249-a020-d8a0f2edb516 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:52:12.327884 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6adc0015-f7a7-405f-9958-20fa2e8d3de3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:52:12.328478 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6adc0015-f7a7-405f-9958-20fa2e8d3de3 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:12.328753 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6adc0015-f7a7-405f-9958-20fa2e8d3de3 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:12.329018 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6adc0015-f7a7-405f-9958-20fa2e8d3de3 None admin] PATH_INFO: `/v3/roles` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:12.329704 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6adc0015-f7a7-405f-9958-20fa2e8d3de3 None admin] RBAC: Adding query filter params (name=service) {{(pid=81423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 10:52:12.330024 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6adc0015-f7a7-405f-9958-20fa2e8d3de3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:52:12.331360 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6adc0015-f7a7-405f-9958-20fa2e8d3de3 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:52:12.333174 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application [None req-dc218a41-b40a-4249-a020-d8a0f2edb516 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 10:52:12.333174 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 10:52:12.333174 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 10:52:12.333174 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 10:52:12.333174 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 10:52:12.333174 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 10:52:12.333174 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 10:52:12.333174 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 10:52:12.333174 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 10:52:12.333174 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 10:52:12.333174 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 10:52:12.333174 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 10:52:12.333174 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 26 10:52:12.333174 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 26 10:52:12.333174 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 26 10:52:12.334091 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 26 10:52:12.334091 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 10:52:12.334091 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 10:52:12.334091 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 10:52:12.334091 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 10:52:12.334091 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 10:52:12.334091 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 10:52:12.334091 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 10:52:12.334091 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application with Lock( Aug 26 10:52:12.334091 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 10:52:12.334091 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._enter() Aug 26 10:52:12.334091 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 10:52:12.334091 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 10:52:12.334091 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 10:52:12.334091 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.creator() Aug 26 10:52:12.334091 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 10:52:12.334091 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application created_value = creator( Aug 26 10:52:12.335007 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 26 10:52:12.335007 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 26 10:52:12.335007 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 26 10:52:12.335007 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 26 10:52:12.335007 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 26 10:52:12.335007 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 26 10:52:12.335007 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 10:52:12.335007 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Aug 26 10:52:12.335007 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 284/569] 10.222.0.40 () {56 vars in 1111 bytes} [Fri Aug 26 10:52:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 10:52:12.335486 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc218a41-b40a-4249-a020-d8a0f2edb516 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:52:12.346689 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 286/570] 10.222.0.40 () {56 vars in 1103 bytes} [Fri Aug 26 10:52:12 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 10:52:12.346689 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6adc0015-f7a7-405f-9958-20fa2e8d3de3 None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:52:12.437755 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc218a41-b40a-4249-a020-d8a0f2edb516 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:52:12.440768 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6adc0015-f7a7-405f-9958-20fa2e8d3de3 None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:52:12.452971 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:52:12.472849 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20398c4a-86bf-4bbc-9dfa-6686c2121c65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:52:12.473477 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20398c4a-86bf-4bbc-9dfa-6686c2121c65 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:12.473759 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20398c4a-86bf-4bbc-9dfa-6686c2121c65 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:12.474063 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20398c4a-86bf-4bbc-9dfa-6686c2121c65 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:12.474952 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9e0f2d-2b95-4612-8329-0a89fabba28a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:52:12.475565 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9e0f2d-2b95-4612-8329-0a89fabba28a None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:12.475838 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9e0f2d-2b95-4612-8329-0a89fabba28a None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:12.476101 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9e0f2d-2b95-4612-8329-0a89fabba28a None admin] PATH_INFO: `/v3/users/glance` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:12.481473 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20398c4a-86bf-4bbc-9dfa-6686c2121c65 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:52:12.482949 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20398c4a-86bf-4bbc-9dfa-6686c2121c65 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:52:12.483516 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d9e0f2d-2b95-4612-8329-0a89fabba28a None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:52:12.484911 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d9e0f2d-2b95-4612-8329-0a89fabba28a None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:52:12.494447 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application [None req-20398c4a-86bf-4bbc-9dfa-6686c2121c65 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 26 10:52:12.494447 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 10:52:12.494447 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 10:52:12.494447 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 10:52:12.494447 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 10:52:12.494447 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 10:52:12.494447 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 10:52:12.494447 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 10:52:12.494447 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 10:52:12.494447 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 10:52:12.494447 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 10:52:12.494447 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 10:52:12.494447 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 26 10:52:12.494447 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 26 10:52:12.494447 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 26 10:52:12.494447 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 26 10:52:12.495546 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 10:52:12.495546 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 10:52:12.495546 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 26 10:52:12.495546 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 10:52:12.495546 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 26 10:52:12.495546 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 10:52:12.495546 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 10:52:12.495546 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 10:52:12.495546 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 10:52:12.495546 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 10:52:12.495546 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 10:52:12.495546 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application with Lock( Aug 26 10:52:12.495546 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 10:52:12.495546 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._enter() Aug 26 10:52:12.495546 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 10:52:12.495546 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 10:52:12.495546 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 10:52:12.495546 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.creator() Aug 26 10:52:12.502213 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 10:52:12.502213 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application created_value = creator( Aug 26 10:52:12.502213 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 26 10:52:12.502213 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 26 10:52:12.502213 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 26 10:52:12.502213 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 26 10:52:12.502213 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 26 10:52:12.502213 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 26 10:52:12.502213 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 26 10:52:12.502213 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Aug 26 10:52:12.502213 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 285/571] 10.222.0.40 () {56 vars in 1093 bytes} [Fri Aug 26 10:52:12 2022] GET /identity/v3/users/nova => generated 82 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 10:52:12.503181 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application [None req-8d9e0f2d-2b95-4612-8329-0a89fabba28a None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 26 10:52:12.503181 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 10:52:12.503181 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 10:52:12.503181 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 10:52:12.503181 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 10:52:12.503181 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 10:52:12.503181 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 10:52:12.503181 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 10:52:12.503181 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 10:52:12.503181 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 10:52:12.503181 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 10:52:12.503181 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 10:52:12.503181 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 26 10:52:12.503181 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 26 10:52:12.503181 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 26 10:52:12.503181 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 26 10:52:12.504521 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 10:52:12.504521 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 10:52:12.504521 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 26 10:52:12.504521 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 10:52:12.504521 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 26 10:52:12.504521 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 10:52:12.504521 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 10:52:12.504521 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 10:52:12.504521 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 10:52:12.504521 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 10:52:12.504521 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 10:52:12.504521 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application with Lock( Aug 26 10:52:12.504521 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 10:52:12.504521 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._enter() Aug 26 10:52:12.504521 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 10:52:12.504521 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 10:52:12.504521 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 10:52:12.504521 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.creator() Aug 26 10:52:12.506801 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 10:52:12.506801 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application created_value = creator( Aug 26 10:52:12.506801 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 26 10:52:12.506801 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 26 10:52:12.506801 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 26 10:52:12.506801 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 26 10:52:12.506801 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 26 10:52:12.506801 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 26 10:52:12.506801 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 26 10:52:12.506801 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Aug 26 10:52:12.507459 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50712a4c-7bae-4557-85f4-015bf4955f64 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:12.507544 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 287/572] 10.222.0.40 () {56 vars in 1099 bytes} [Fri Aug 26 10:52:12 2022] GET /identity/v3/users/glance => generated 84 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 10:52:12.507624 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50712a4c-7bae-4557-85f4-015bf4955f64 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:12.507624 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50712a4c-7bae-4557-85f4-015bf4955f64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:12.507779 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9e0f2d-2b95-4612-8329-0a89fabba28a None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:52:12.517448 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-50712a4c-7bae-4557-85f4-015bf4955f64 None None] MFA Rules not processed for user `ca155a48e2024c32af3fd2e19e5fac11`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 10:52:12.519199 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-50712a4c-7bae-4557-85f4-015bf4955f64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 10:52:12.521904 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:52:12.587834 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 286/573] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 26 10:52:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 10:52:12.588669 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50712a4c-7bae-4557-85f4-015bf4955f64 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:52:12.601890 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9e0f2d-2b95-4612-8329-0a89fabba28a None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:52:12.636316 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1555cdf-87b9-41e8-bbfa-44255eb3468a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:52:12.636917 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1555cdf-87b9-41e8-bbfa-44255eb3468a None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:12.637195 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1555cdf-87b9-41e8-bbfa-44255eb3468a None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:12.637488 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1555cdf-87b9-41e8-bbfa-44255eb3468a None admin] PATH_INFO: `/v3/projects` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:12.638171 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1555cdf-87b9-41e8-bbfa-44255eb3468a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 10:52:12.638472 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1555cdf-87b9-41e8-bbfa-44255eb3468a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:52:12.639885 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1555cdf-87b9-41e8-bbfa-44255eb3468a None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:52:12.652622 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 288/574] 10.222.0.40 () {56 vars in 1148 bytes} [Fri Aug 26 10:52:12 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 10:52:12.653422 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1555cdf-87b9-41e8-bbfa-44255eb3468a None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:52:12.687856 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:52:12.694125 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50712a4c-7bae-4557-85f4-015bf4955f64 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:52:12.731838 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8de7e1f-2b54-4ed5-9a39-da842c8843bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:52:12.732474 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8de7e1f-2b54-4ed5-9a39-da842c8843bc None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:12.732755 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8de7e1f-2b54-4ed5-9a39-da842c8843bc None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:12.733284 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8de7e1f-2b54-4ed5-9a39-da842c8843bc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:12.743270 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:52:12.745459 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8de7e1f-2b54-4ed5-9a39-da842c8843bc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:52:12.747201 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8de7e1f-2b54-4ed5-9a39-da842c8843bc None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:52:12.762361 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1555cdf-87b9-41e8-bbfa-44255eb3468a None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:52:12.765840 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application [None req-f8de7e1f-2b54-4ed5-9a39-da842c8843bc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 10:52:12.765840 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 10:52:12.765840 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 10:52:12.765840 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 10:52:12.765840 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 10:52:12.765840 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 10:52:12.765840 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 10:52:12.765840 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 10:52:12.765840 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 10:52:12.765840 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 10:52:12.765840 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 10:52:12.765840 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 10:52:12.765840 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 10:52:12.765840 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 10:52:12.765840 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 10:52:12.765840 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 10:52:12.766933 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 10:52:12.766933 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 10:52:12.766933 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 10:52:12.766933 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 10:52:12.766933 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 10:52:12.766933 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 10:52:12.766933 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 10:52:12.766933 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application with Lock( Aug 26 10:52:12.766933 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 10:52:12.766933 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._enter() Aug 26 10:52:12.766933 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 10:52:12.766933 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 10:52:12.766933 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 10:52:12.766933 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.creator() Aug 26 10:52:12.766933 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 10:52:12.766933 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application created_value = creator( Aug 26 10:52:12.766933 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 10:52:12.773441 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 10:52:12.773441 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 10:52:12.773441 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Aug 26 10:52:12.773441 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 287/575] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Aug 26 10:52:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 10:52:12.773441 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8de7e1f-2b54-4ed5-9a39-da842c8843bc None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:52:12.801234 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2645aa38-281a-4e73-a086-1bd85cde9d9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:52:12.801839 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2645aa38-281a-4e73-a086-1bd85cde9d9a None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:12.802172 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2645aa38-281a-4e73-a086-1bd85cde9d9a None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:12.802446 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2645aa38-281a-4e73-a086-1bd85cde9d9a None admin] PATH_INFO: `/v3/users` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:12.803166 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2645aa38-281a-4e73-a086-1bd85cde9d9a None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 10:52:12.803466 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2645aa38-281a-4e73-a086-1bd85cde9d9a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:52:12.804884 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2645aa38-281a-4e73-a086-1bd85cde9d9a None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:52:12.827299 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 289/576] 10.222.0.40 () {56 vars in 1133 bytes} [Fri Aug 26 10:52:12 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 10:52:12.827299 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2645aa38-281a-4e73-a086-1bd85cde9d9a None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:52:12.886616 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8de7e1f-2b54-4ed5-9a39-da842c8843bc None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:52:12.920395 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521f0795-9a2b-45ea-b7ad-8def62a32be9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:52:12.920992 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521f0795-9a2b-45ea-b7ad-8def62a32be9 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:12.921265 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521f0795-9a2b-45ea-b7ad-8def62a32be9 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:12.921530 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521f0795-9a2b-45ea-b7ad-8def62a32be9 None admin] PATH_INFO: `/v3/users` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:12.922287 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-521f0795-9a2b-45ea-b7ad-8def62a32be9 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 10:52:12.922586 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-521f0795-9a2b-45ea-b7ad-8def62a32be9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:52:12.923872 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:52:12.925132 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-521f0795-9a2b-45ea-b7ad-8def62a32be9 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:52:12.933914 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2645aa38-281a-4e73-a086-1bd85cde9d9a None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:52:12.942921 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 288/577] 10.222.0.40 () {56 vars in 1137 bytes} [Fri Aug 26 10:52:12 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 10:52:12.943531 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521f0795-9a2b-45ea-b7ad-8def62a32be9 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:52:12.972000 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f818cf43-8193-4904-b5f4-908465c483ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:52:12.972602 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f818cf43-8193-4904-b5f4-908465c483ed None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:12.972878 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f818cf43-8193-4904-b5f4-908465c483ed None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:12.973147 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f818cf43-8193-4904-b5f4-908465c483ed None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:12.973865 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f818cf43-8193-4904-b5f4-908465c483ed None admin] RBAC: Adding query filter params (role.id=3bda770fcd5c46cdbfabcd7404a25e66, scope.project.id=0e30ccf0c1204727b786876c141dbcd2, user.id=6529bea296c9482a973ced5fe9f2212f) {{(pid=81423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 10:52:12.974204 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f818cf43-8193-4904-b5f4-908465c483ed None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:52:12.975634 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f818cf43-8193-4904-b5f4-908465c483ed None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:52:12.980835 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 290/578] 10.222.0.40 () {56 vars in 1374 bytes} [Fri Aug 26 10:52:12 2022] GET /identity/v3/role_assignments?user.id=6529bea296c9482a973ced5fe9f2212f&scope.project.id=0e30ccf0c1204727b786876c141dbcd2&role.id=3bda770fcd5c46cdbfabcd7404a25e66 => generated 259 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 10:52:12.981602 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f818cf43-8193-4904-b5f4-908465c483ed None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:52:12.996177 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:52:13.059165 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521f0795-9a2b-45ea-b7ad-8def62a32be9 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:52:13.090593 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f818cf43-8193-4904-b5f4-908465c483ed None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:52:13.098196 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79dae20d-a732-4bba-8cd6-df3e9b29118d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:52:13.098950 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79dae20d-a732-4bba-8cd6-df3e9b29118d None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:13.099238 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79dae20d-a732-4bba-8cd6-df3e9b29118d None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:13.099515 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79dae20d-a732-4bba-8cd6-df3e9b29118d None admin] PATH_INFO: `/v3/domains` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:13.100211 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79dae20d-a732-4bba-8cd6-df3e9b29118d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 10:52:13.100517 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79dae20d-a732-4bba-8cd6-df3e9b29118d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:52:13.102052 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79dae20d-a732-4bba-8cd6-df3e9b29118d None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:52:13.120035 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 289/579] 10.222.0.40 () {56 vars in 1109 bytes} [Fri Aug 26 10:52:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 10:52:13.120035 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79dae20d-a732-4bba-8cd6-df3e9b29118d None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:52:13.134427 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b30cc8d-6f5a-466c-90bf-a0fe400604d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:52:13.135153 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b30cc8d-6f5a-466c-90bf-a0fe400604d4 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:13.135508 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b30cc8d-6f5a-466c-90bf-a0fe400604d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:13.135847 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b30cc8d-6f5a-466c-90bf-a0fe400604d4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:13.147033 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b30cc8d-6f5a-466c-90bf-a0fe400604d4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:52:13.148574 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b30cc8d-6f5a-466c-90bf-a0fe400604d4 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:52:13.165730 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application [None req-6b30cc8d-6f5a-466c-90bf-a0fe400604d4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 10:52:13.165730 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 10:52:13.165730 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 10:52:13.165730 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 10:52:13.165730 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 10:52:13.165730 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 10:52:13.165730 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 10:52:13.165730 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 10:52:13.165730 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 10:52:13.165730 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 10:52:13.165730 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 10:52:13.165730 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 10:52:13.165730 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 10:52:13.165730 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 10:52:13.165730 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 10:52:13.165730 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 10:52:13.166745 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 10:52:13.166745 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 10:52:13.166745 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 10:52:13.166745 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 10:52:13.166745 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 10:52:13.166745 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 10:52:13.166745 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 10:52:13.166745 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application with Lock( Aug 26 10:52:13.166745 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 10:52:13.166745 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._enter() Aug 26 10:52:13.166745 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 10:52:13.166745 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 10:52:13.166745 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 10:52:13.166745 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.creator() Aug 26 10:52:13.166745 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 10:52:13.166745 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application created_value = creator( Aug 26 10:52:13.166745 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 10:52:13.167634 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 10:52:13.167634 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 10:52:13.167634 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Aug 26 10:52:13.167634 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 291/580] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Aug 26 10:52:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 10:52:13.167867 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b30cc8d-6f5a-466c-90bf-a0fe400604d4 None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:52:13.180967 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:52:13.233675 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79dae20d-a732-4bba-8cd6-df3e9b29118d None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:52:13.244629 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:52:13.271834 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f6a5b8-f8f2-4d1a-bb7a-0601cf8c7d2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:52:13.272378 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f6a5b8-f8f2-4d1a-bb7a-0601cf8c7d2b None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:13.272654 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f6a5b8-f8f2-4d1a-bb7a-0601cf8c7d2b None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:13.272923 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f6a5b8-f8f2-4d1a-bb7a-0601cf8c7d2b None admin] PATH_INFO: `/v3/roles/service` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:13.284069 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f6a5b8-f8f2-4d1a-bb7a-0601cf8c7d2b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 10:52:13.288839 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b30cc8d-6f5a-466c-90bf-a0fe400604d4 None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:52:13.289222 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application [None req-30f6a5b8-f8f2-4d1a-bb7a-0601cf8c7d2b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 26 10:52:13.289222 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 10:52:13.289222 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 10:52:13.289222 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 10:52:13.289222 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 10:52:13.289222 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 10:52:13.289222 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 10:52:13.289222 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 10:52:13.289222 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 10:52:13.289222 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 10:52:13.289222 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 10:52:13.289222 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 10:52:13.289222 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 10:52:13.289222 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 10:52:13.289222 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 10:52:13.289222 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 10:52:13.290166 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 10:52:13.290166 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 10:52:13.290166 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 10:52:13.290166 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 10:52:13.290166 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 10:52:13.290166 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 10:52:13.290166 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 10:52:13.290166 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 10:52:13.290166 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 10:52:13.290166 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 10:52:13.290166 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 10:52:13.290166 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application with Lock( Aug 26 10:52:13.290166 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 10:52:13.290166 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._enter() Aug 26 10:52:13.290166 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 10:52:13.290166 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 10:52:13.291097 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 10:52:13.291097 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.creator() Aug 26 10:52:13.291097 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 10:52:13.291097 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application created_value = creator( Aug 26 10:52:13.291097 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 10:52:13.291097 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 10:52:13.291097 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 10:52:13.291097 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 10:52:13.291097 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 10:52:13.291097 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 10:52:13.291097 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 26 10:52:13.291097 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Aug 26 10:52:13.291097 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 290/581] 10.222.0.40 () {56 vars in 1102 bytes} [Fri Aug 26 10:52:13 2022] GET /identity/v3/roles/service => generated 85 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 10:52:13.291778 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f6a5b8-f8f2-4d1a-bb7a-0601cf8c7d2b None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:52:13.331529 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe09c426-dd87-4b9b-9d5a-97d58655dd88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:52:13.332138 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe09c426-dd87-4b9b-9d5a-97d58655dd88 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:13.332413 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe09c426-dd87-4b9b-9d5a-97d58655dd88 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:13.332683 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe09c426-dd87-4b9b-9d5a-97d58655dd88 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:13.343289 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe09c426-dd87-4b9b-9d5a-97d58655dd88 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:52:13.344730 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe09c426-dd87-4b9b-9d5a-97d58655dd88 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:52:13.360336 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application [None req-fe09c426-dd87-4b9b-9d5a-97d58655dd88 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 10:52:13.360336 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 10:52:13.360336 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 10:52:13.360336 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 10:52:13.360336 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 10:52:13.360336 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 10:52:13.360336 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 10:52:13.360336 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 10:52:13.360336 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 10:52:13.360336 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 10:52:13.360336 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 10:52:13.360336 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 10:52:13.360336 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 10:52:13.360336 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 10:52:13.360336 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 10:52:13.360336 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 10:52:13.361728 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 10:52:13.361728 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 10:52:13.361728 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 10:52:13.361728 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 10:52:13.361728 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 10:52:13.361728 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 10:52:13.361728 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 10:52:13.361728 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application with Lock( Aug 26 10:52:13.361728 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 10:52:13.361728 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._enter() Aug 26 10:52:13.361728 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 10:52:13.361728 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 10:52:13.361728 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 10:52:13.361728 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.creator() Aug 26 10:52:13.361728 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 10:52:13.361728 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application created_value = creator( Aug 26 10:52:13.361728 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 10:52:13.362979 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 10:52:13.362979 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 10:52:13.362979 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Aug 26 10:52:13.362979 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 292/582] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Aug 26 10:52:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 10:52:13.362979 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe09c426-dd87-4b9b-9d5a-97d58655dd88 None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:52:13.411983 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f6a5b8-f8f2-4d1a-bb7a-0601cf8c7d2b None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:52:13.424725 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:52:13.451879 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86be9c31-5ac0-4c77-91a9-5360e1cdb598 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:52:13.452512 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86be9c31-5ac0-4c77-91a9-5360e1cdb598 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:13.452791 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86be9c31-5ac0-4c77-91a9-5360e1cdb598 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:13.453038 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86be9c31-5ac0-4c77-91a9-5360e1cdb598 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:13.464105 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86be9c31-5ac0-4c77-91a9-5360e1cdb598 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:52:13.465824 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86be9c31-5ac0-4c77-91a9-5360e1cdb598 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:52:13.473542 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe09c426-dd87-4b9b-9d5a-97d58655dd88 None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:52:13.479003 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application [None req-86be9c31-5ac0-4c77-91a9-5360e1cdb598 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 26 10:52:13.479003 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 10:52:13.479003 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 10:52:13.479003 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 10:52:13.479003 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 10:52:13.479003 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 10:52:13.479003 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 10:52:13.479003 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 10:52:13.479003 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 10:52:13.479003 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 10:52:13.479003 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 10:52:13.479003 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 10:52:13.479003 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 26 10:52:13.479003 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 26 10:52:13.479003 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 26 10:52:13.479003 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 26 10:52:13.479907 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 10:52:13.479907 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 10:52:13.479907 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 26 10:52:13.479907 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 10:52:13.479907 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 26 10:52:13.479907 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 10:52:13.479907 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 10:52:13.479907 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 10:52:13.479907 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 10:52:13.479907 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 10:52:13.479907 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 10:52:13.479907 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application with Lock( Aug 26 10:52:13.479907 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 10:52:13.479907 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._enter() Aug 26 10:52:13.479907 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 10:52:13.479907 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 10:52:13.479907 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 10:52:13.479907 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.creator() Aug 26 10:52:13.480839 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 10:52:13.480839 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application created_value = creator( Aug 26 10:52:13.480839 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 26 10:52:13.480839 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 26 10:52:13.480839 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 26 10:52:13.480839 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 26 10:52:13.480839 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 26 10:52:13.480839 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 26 10:52:13.480839 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 26 10:52:13.480839 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Aug 26 10:52:13.484995 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 291/583] 10.222.0.40 () {56 vars in 1099 bytes} [Fri Aug 26 10:52:13 2022] GET /identity/v3/users/cinder => generated 84 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 10:52:13.485874 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86be9c31-5ac0-4c77-91a9-5360e1cdb598 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:52:13.509605 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459fbda5-edba-41c9-b20d-ee1a469c5861 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:52:13.510659 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459fbda5-edba-41c9-b20d-ee1a469c5861 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:13.510943 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459fbda5-edba-41c9-b20d-ee1a469c5861 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:13.511213 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459fbda5-edba-41c9-b20d-ee1a469c5861 None admin] PATH_INFO: `/v3/domains` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:13.512125 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-459fbda5-edba-41c9-b20d-ee1a469c5861 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 10:52:13.512429 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-459fbda5-edba-41c9-b20d-ee1a469c5861 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:52:13.514299 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-459fbda5-edba-41c9-b20d-ee1a469c5861 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:52:13.519625 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:52:13.533844 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 293/584] 10.222.0.40 () {56 vars in 1109 bytes} [Fri Aug 26 10:52:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 10:52:13.533844 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459fbda5-edba-41c9-b20d-ee1a469c5861 None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:52:13.597124 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86be9c31-5ac0-4c77-91a9-5360e1cdb598 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:52:13.636931 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f6a652-3e58-42b4-8c44-08fad3185f55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:52:13.637534 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f6a652-3e58-42b4-8c44-08fad3185f55 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:13.637809 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f6a652-3e58-42b4-8c44-08fad3185f55 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:13.638110 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f6a652-3e58-42b4-8c44-08fad3185f55 None admin] PATH_INFO: `/v3/roles` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:13.638710 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459fbda5-edba-41c9-b20d-ee1a469c5861 None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:52:13.638820 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f6a652-3e58-42b4-8c44-08fad3185f55 None admin] RBAC: Adding query filter params (name=service) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 10:52:13.639081 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f6a652-3e58-42b4-8c44-08fad3185f55 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:52:13.640432 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f6a652-3e58-42b4-8c44-08fad3185f55 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:52:13.649103 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 292/585] 10.222.0.40 () {56 vars in 1103 bytes} [Fri Aug 26 10:52:13 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 10:52:13.651233 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f6a652-3e58-42b4-8c44-08fad3185f55 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:52:13.657548 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:52:13.675154 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407ff9c0-29fe-48b1-bc46-12497c30e486 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:52:13.675759 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407ff9c0-29fe-48b1-bc46-12497c30e486 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:13.676037 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407ff9c0-29fe-48b1-bc46-12497c30e486 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:13.676312 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407ff9c0-29fe-48b1-bc46-12497c30e486 None admin] PATH_INFO: `/v3/domains` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:13.676948 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-407ff9c0-29fe-48b1-bc46-12497c30e486 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 10:52:13.677249 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-407ff9c0-29fe-48b1-bc46-12497c30e486 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:52:13.678642 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-407ff9c0-29fe-48b1-bc46-12497c30e486 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:52:13.690464 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 294/586] 10.222.0.40 () {56 vars in 1109 bytes} [Fri Aug 26 10:52:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 10:52:13.691335 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407ff9c0-29fe-48b1-bc46-12497c30e486 None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:52:13.759003 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:52:13.769501 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f6a652-3e58-42b4-8c44-08fad3185f55 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:52:13.791286 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407ff9c0-29fe-48b1-bc46-12497c30e486 None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:52:13.803771 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67fb6c7c-50df-4730-b56c-4e0da1d57a7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:52:13.804656 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67fb6c7c-50df-4730-b56c-4e0da1d57a7c None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:13.805196 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67fb6c7c-50df-4730-b56c-4e0da1d57a7c None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:13.805470 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67fb6c7c-50df-4730-b56c-4e0da1d57a7c None admin] PATH_INFO: `/v3/users` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:13.806242 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67fb6c7c-50df-4730-b56c-4e0da1d57a7c None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 10:52:13.806550 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67fb6c7c-50df-4730-b56c-4e0da1d57a7c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:52:13.808105 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67fb6c7c-50df-4730-b56c-4e0da1d57a7c None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:52:13.833683 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 293/587] 10.222.0.40 () {56 vars in 1137 bytes} [Fri Aug 26 10:52:13 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 10:52:13.833683 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67fb6c7c-50df-4730-b56c-4e0da1d57a7c None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:52:13.835170 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6411bd-d7bd-49c6-9468-ba21a5af6275 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:52:13.835849 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6411bd-d7bd-49c6-9468-ba21a5af6275 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:13.836125 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6411bd-d7bd-49c6-9468-ba21a5af6275 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:13.836388 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6411bd-d7bd-49c6-9468-ba21a5af6275 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:13.845316 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d6411bd-d7bd-49c6-9468-ba21a5af6275 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:52:13.847607 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d6411bd-d7bd-49c6-9468-ba21a5af6275 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:52:13.859656 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application [None req-2d6411bd-d7bd-49c6-9468-ba21a5af6275 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 10:52:13.859656 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 10:52:13.859656 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 10:52:13.859656 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 10:52:13.859656 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 10:52:13.859656 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 10:52:13.859656 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 10:52:13.859656 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 10:52:13.859656 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 10:52:13.859656 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 10:52:13.859656 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 10:52:13.859656 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 10:52:13.859656 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 26 10:52:13.859656 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 26 10:52:13.859656 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 26 10:52:13.860616 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 26 10:52:13.860616 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 10:52:13.860616 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 10:52:13.860616 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 10:52:13.860616 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 10:52:13.860616 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 10:52:13.860616 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 10:52:13.860616 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 10:52:13.860616 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application with Lock( Aug 26 10:52:13.860616 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 10:52:13.860616 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._enter() Aug 26 10:52:13.860616 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 10:52:13.860616 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 10:52:13.860616 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 10:52:13.860616 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.creator() Aug 26 10:52:13.860616 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 10:52:13.860616 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application created_value = creator( Aug 26 10:52:13.861509 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 26 10:52:13.861509 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 26 10:52:13.861509 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 26 10:52:13.861509 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 26 10:52:13.861509 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 26 10:52:13.861509 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 26 10:52:13.861509 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 10:52:13.861509 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Aug 26 10:52:13.861509 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 295/588] 10.222.0.40 () {56 vars in 1111 bytes} [Fri Aug 26 10:52:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 10:52:13.862026 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6411bd-d7bd-49c6-9468-ba21a5af6275 None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:52:13.911502 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:52:13.949085 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67fb6c7c-50df-4730-b56c-4e0da1d57a7c None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:52:13.983712 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71d45c6-bb32-404b-9593-e65379a1c2f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:52:13.984053 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6411bd-d7bd-49c6-9468-ba21a5af6275 None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:52:13.984482 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71d45c6-bb32-404b-9593-e65379a1c2f6 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:13.984588 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71d45c6-bb32-404b-9593-e65379a1c2f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:13.984851 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71d45c6-bb32-404b-9593-e65379a1c2f6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:13.997138 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c71d45c6-bb32-404b-9593-e65379a1c2f6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:52:13.998613 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c71d45c6-bb32-404b-9593-e65379a1c2f6 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:52:14.014158 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application [None req-c71d45c6-bb32-404b-9593-e65379a1c2f6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 10:52:14.014158 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 10:52:14.014158 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 10:52:14.014158 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 10:52:14.014158 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 10:52:14.014158 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 10:52:14.014158 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 10:52:14.014158 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 10:52:14.014158 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 10:52:14.014158 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 10:52:14.014158 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 10:52:14.014158 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 10:52:14.014158 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 10:52:14.014158 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 10:52:14.014158 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 10:52:14.014158 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 10:52:14.015375 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 10:52:14.015375 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 10:52:14.015375 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 10:52:14.015375 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 10:52:14.015375 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 10:52:14.015375 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 10:52:14.015375 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 10:52:14.015375 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application with Lock( Aug 26 10:52:14.015375 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 10:52:14.015375 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._enter() Aug 26 10:52:14.015375 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 10:52:14.015375 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 10:52:14.015375 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 10:52:14.015375 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.creator() Aug 26 10:52:14.015375 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 10:52:14.015375 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application created_value = creator( Aug 26 10:52:14.015375 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 10:52:14.016491 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 10:52:14.016491 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 10:52:14.016491 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Aug 26 10:52:14.016491 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 294/589] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Aug 26 10:52:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 10:52:14.016491 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71d45c6-bb32-404b-9593-e65379a1c2f6 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:52:14.019087 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:52:14.028167 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662ad32a-1f0f-48ac-a524-08ff0d8e83cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:52:14.029231 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662ad32a-1f0f-48ac-a524-08ff0d8e83cb None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:14.029573 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662ad32a-1f0f-48ac-a524-08ff0d8e83cb None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:14.029909 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662ad32a-1f0f-48ac-a524-08ff0d8e83cb None admin] PATH_INFO: `/v3/projects/service` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:14.037491 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-662ad32a-1f0f-48ac-a524-08ff0d8e83cb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:52:14.038959 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-662ad32a-1f0f-48ac-a524-08ff0d8e83cb None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:52:14.052024 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application [None req-662ad32a-1f0f-48ac-a524-08ff0d8e83cb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 10:52:14.052024 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 10:52:14.052024 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 10:52:14.052024 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 10:52:14.052024 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 10:52:14.052024 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 10:52:14.052024 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 10:52:14.052024 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 10:52:14.052024 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 10:52:14.052024 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 10:52:14.052024 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 10:52:14.052024 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 10:52:14.052024 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 26 10:52:14.052024 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 26 10:52:14.052024 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 26 10:52:14.053042 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 26 10:52:14.053042 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 10:52:14.053042 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 10:52:14.053042 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 10:52:14.053042 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 10:52:14.053042 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 10:52:14.053042 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 10:52:14.053042 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 10:52:14.053042 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application with Lock( Aug 26 10:52:14.053042 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 10:52:14.053042 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._enter() Aug 26 10:52:14.053042 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 10:52:14.053042 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 10:52:14.053042 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 10:52:14.053042 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.creator() Aug 26 10:52:14.053042 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 10:52:14.053042 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application created_value = creator( Aug 26 10:52:14.053917 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 26 10:52:14.053917 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 26 10:52:14.053917 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 26 10:52:14.053917 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 26 10:52:14.053917 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 26 10:52:14.053917 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 26 10:52:14.053917 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 10:52:14.053917 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Aug 26 10:52:14.053917 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 296/590] 10.222.0.40 () {56 vars in 1111 bytes} [Fri Aug 26 10:52:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 10:52:14.054426 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662ad32a-1f0f-48ac-a524-08ff0d8e83cb None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:52:14.132096 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71d45c6-bb32-404b-9593-e65379a1c2f6 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:52:14.166515 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:52:14.171110 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c12081-4d0e-45d4-b2c7-dcab44e0af9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:52:14.171279 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662ad32a-1f0f-48ac-a524-08ff0d8e83cb None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:52:14.171747 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c12081-4d0e-45d4-b2c7-dcab44e0af9e None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:14.172027 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c12081-4d0e-45d4-b2c7-dcab44e0af9e None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:14.172296 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c12081-4d0e-45d4-b2c7-dcab44e0af9e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:14.184671 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c12081-4d0e-45d4-b2c7-dcab44e0af9e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:52:14.186193 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c12081-4d0e-45d4-b2c7-dcab44e0af9e None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:52:14.202131 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application [None req-f5c12081-4d0e-45d4-b2c7-dcab44e0af9e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 10:52:14.202131 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 10:52:14.202131 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 10:52:14.202131 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 10:52:14.202131 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 10:52:14.202131 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 10:52:14.202131 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 10:52:14.202131 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 10:52:14.202131 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 10:52:14.202131 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 10:52:14.202131 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 10:52:14.202131 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 10:52:14.202131 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 10:52:14.202131 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 10:52:14.202131 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 10:52:14.202131 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 10:52:14.203057 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 10:52:14.203057 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 10:52:14.203057 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 10:52:14.203057 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 10:52:14.203057 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 10:52:14.203057 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 10:52:14.203057 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 10:52:14.203057 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application with Lock( Aug 26 10:52:14.203057 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 10:52:14.203057 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._enter() Aug 26 10:52:14.203057 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 10:52:14.203057 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 10:52:14.203057 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 10:52:14.203057 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.creator() Aug 26 10:52:14.203057 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 10:52:14.203057 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application created_value = creator( Aug 26 10:52:14.203057 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 10:52:14.203934 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 10:52:14.203934 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 10:52:14.203934 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Aug 26 10:52:14.203934 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 295/591] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Aug 26 10:52:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 10:52:14.204164 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c12081-4d0e-45d4-b2c7-dcab44e0af9e None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:52:14.217295 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b639d47-457b-4936-9295-2dfb2d9d217c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:52:14.218003 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b639d47-457b-4936-9295-2dfb2d9d217c None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:14.218293 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b639d47-457b-4936-9295-2dfb2d9d217c None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:14.218567 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b639d47-457b-4936-9295-2dfb2d9d217c None admin] PATH_INFO: `/v3/projects` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:14.219266 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b639d47-457b-4936-9295-2dfb2d9d217c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 10:52:14.219571 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b639d47-457b-4936-9295-2dfb2d9d217c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:52:14.221022 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b639d47-457b-4936-9295-2dfb2d9d217c None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:52:14.233527 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 297/592] 10.222.0.40 () {56 vars in 1148 bytes} [Fri Aug 26 10:52:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 10:52:14.234353 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b639d47-457b-4936-9295-2dfb2d9d217c None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:52:14.270234 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:52:14.317449 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c12081-4d0e-45d4-b2c7-dcab44e0af9e None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:52:14.341757 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b639d47-457b-4936-9295-2dfb2d9d217c None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:52:14.355251 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10542051-37d2-4418-acb4-9bc36db770b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:52:14.355867 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10542051-37d2-4418-acb4-9bc36db770b3 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:14.356150 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10542051-37d2-4418-acb4-9bc36db770b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:14.356426 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10542051-37d2-4418-acb4-9bc36db770b3 None admin] PATH_INFO: `/v3/domains` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:14.357088 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10542051-37d2-4418-acb4-9bc36db770b3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 10:52:14.357398 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10542051-37d2-4418-acb4-9bc36db770b3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:52:14.359054 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10542051-37d2-4418-acb4-9bc36db770b3 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:52:14.371903 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 296/593] 10.222.0.40 () {56 vars in 1109 bytes} [Fri Aug 26 10:52:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 10:52:14.372519 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10542051-37d2-4418-acb4-9bc36db770b3 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:52:14.384552 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b75ad0e-1a48-4a8a-9bee-4aa75a9ca441 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:52:14.384960 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b75ad0e-1a48-4a8a-9bee-4aa75a9ca441 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:14.385266 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b75ad0e-1a48-4a8a-9bee-4aa75a9ca441 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:14.385570 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b75ad0e-1a48-4a8a-9bee-4aa75a9ca441 None admin] PATH_INFO: `/v3/projects` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:14.386233 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b75ad0e-1a48-4a8a-9bee-4aa75a9ca441 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 10:52:14.386540 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b75ad0e-1a48-4a8a-9bee-4aa75a9ca441 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:52:14.387958 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b75ad0e-1a48-4a8a-9bee-4aa75a9ca441 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:52:14.399856 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 298/594] 10.222.0.40 () {56 vars in 1148 bytes} [Fri Aug 26 10:52:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 10:52:14.400661 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b75ad0e-1a48-4a8a-9bee-4aa75a9ca441 None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:52:14.422025 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:52:14.487170 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10542051-37d2-4418-acb4-9bc36db770b3 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:52:14.506268 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:52:14.522051 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b75ad0e-1a48-4a8a-9bee-4aa75a9ca441 None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:52:14.529480 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5966ef7-d1d4-41ab-94b9-a36ba0789e54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:52:14.530534 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5966ef7-d1d4-41ab-94b9-a36ba0789e54 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:14.530828 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5966ef7-d1d4-41ab-94b9-a36ba0789e54 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:14.531102 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5966ef7-d1d4-41ab-94b9-a36ba0789e54 None admin] PATH_INFO: `/v3/domains` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:14.531754 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5966ef7-d1d4-41ab-94b9-a36ba0789e54 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 10:52:14.532058 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5966ef7-d1d4-41ab-94b9-a36ba0789e54 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:52:14.534452 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5966ef7-d1d4-41ab-94b9-a36ba0789e54 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:52:14.547861 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 297/595] 10.222.0.40 () {56 vars in 1109 bytes} [Fri Aug 26 10:52:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 10:52:14.549234 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5966ef7-d1d4-41ab-94b9-a36ba0789e54 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:52:14.564987 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee72f6d-8fe4-4a48-b91d-383d08db90b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:52:14.565838 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee72f6d-8fe4-4a48-b91d-383d08db90b4 None admin] REQUEST_METHOD: `PUT` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:14.565969 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee72f6d-8fe4-4a48-b91d-383d08db90b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:14.566212 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee72f6d-8fe4-4a48-b91d-383d08db90b4 None admin] PATH_INFO: `/v3/projects/0e30ccf0c1204727b786876c141dbcd2/users/1f4b4a662db049ecacc865ecff10bdb0/roles/3bda770fcd5c46cdbfabcd7404a25e66` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:14.583559 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee72f6d-8fe4-4a48-b91d-383d08db90b4 None admin] RBAC: Authorizing `identity:create_grant(project_id=0e30ccf0c1204727b786876c141dbcd2, user_id=1f4b4a662db049ecacc865ecff10bdb0, role_id=3bda770fcd5c46cdbfabcd7404a25e66)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:52:14.584945 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee72f6d-8fe4-4a48-b91d-383d08db90b4 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:52:14.598995 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 299/596] 10.222.0.40 () {58 vars in 1439 bytes} [Fri Aug 26 10:52:14 2022] PUT /identity/v3/projects/0e30ccf0c1204727b786876c141dbcd2/users/1f4b4a662db049ecacc865ecff10bdb0/roles/3bda770fcd5c46cdbfabcd7404a25e66 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 10:52:14.599561 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee72f6d-8fe4-4a48-b91d-383d08db90b4 None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:52:14.677435 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5966ef7-d1d4-41ab-94b9-a36ba0789e54 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:52:14.707144 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:52:14.721383 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de1a717-3bd0-4088-834a-18bad5ae13e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:52:14.722115 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de1a717-3bd0-4088-834a-18bad5ae13e7 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:14.722396 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de1a717-3bd0-4088-834a-18bad5ae13e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:14.722670 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de1a717-3bd0-4088-834a-18bad5ae13e7 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:14.731474 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de1a717-3bd0-4088-834a-18bad5ae13e7 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:52:14.733462 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de1a717-3bd0-4088-834a-18bad5ae13e7 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:52:14.745794 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application [None req-0de1a717-3bd0-4088-834a-18bad5ae13e7 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 26 10:52:14.745794 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 10:52:14.745794 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 10:52:14.745794 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 10:52:14.745794 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 10:52:14.745794 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 10:52:14.745794 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 10:52:14.745794 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 10:52:14.745794 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 10:52:14.745794 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 10:52:14.745794 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 10:52:14.745794 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 10:52:14.745794 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 26 10:52:14.745794 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 26 10:52:14.745794 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 26 10:52:14.745794 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 26 10:52:14.746855 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 10:52:14.746855 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 10:52:14.746855 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 26 10:52:14.746855 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 10:52:14.746855 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 26 10:52:14.746855 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 10:52:14.746855 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 10:52:14.746855 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 10:52:14.746855 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 10:52:14.746855 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 10:52:14.746855 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 10:52:14.746855 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application with Lock( Aug 26 10:52:14.746855 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 10:52:14.746855 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._enter() Aug 26 10:52:14.746855 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 10:52:14.746855 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 10:52:14.746855 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 10:52:14.746855 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.creator() Aug 26 10:52:14.747818 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 10:52:14.747818 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application created_value = creator( Aug 26 10:52:14.747818 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 26 10:52:14.747818 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 26 10:52:14.747818 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 26 10:52:14.747818 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 26 10:52:14.747818 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 26 10:52:14.747818 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 26 10:52:14.747818 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 26 10:52:14.747818 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Aug 26 10:52:14.748350 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 298/597] 10.222.0.40 () {56 vars in 1102 bytes} [Fri Aug 26 10:52:14 2022] GET /identity/v3/users/neutron => generated 85 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 10:52:14.749740 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de1a717-3bd0-4088-834a-18bad5ae13e7 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:52:14.754806 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee72f6d-8fe4-4a48-b91d-383d08db90b4 None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:52:14.793918 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:52:14.802422 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c10ad2-742f-4a66-87e8-5509c5482303 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:52:14.802422 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c10ad2-742f-4a66-87e8-5509c5482303 None admin] REQUEST_METHOD: `PUT` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:14.802422 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c10ad2-742f-4a66-87e8-5509c5482303 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:14.802422 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c10ad2-742f-4a66-87e8-5509c5482303 None admin] PATH_INFO: `/v3/projects/0e30ccf0c1204727b786876c141dbcd2/users/8631d7ecfd2849daa8694aecccd22a05/roles/3bda770fcd5c46cdbfabcd7404a25e66` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:14.826258 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c10ad2-742f-4a66-87e8-5509c5482303 None admin] RBAC: Authorizing `identity:create_grant(project_id=0e30ccf0c1204727b786876c141dbcd2, user_id=8631d7ecfd2849daa8694aecccd22a05, role_id=3bda770fcd5c46cdbfabcd7404a25e66)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:52:14.828625 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c10ad2-742f-4a66-87e8-5509c5482303 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:52:14.849041 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 300/598] 10.222.0.40 () {58 vars in 1439 bytes} [Fri Aug 26 10:52:14 2022] PUT /identity/v3/projects/0e30ccf0c1204727b786876c141dbcd2/users/8631d7ecfd2849daa8694aecccd22a05/roles/3bda770fcd5c46cdbfabcd7404a25e66 => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 10:52:14.850296 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c10ad2-742f-4a66-87e8-5509c5482303 None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:52:14.902236 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de1a717-3bd0-4088-834a-18bad5ae13e7 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:52:14.970968 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90218cd5-cbe2-4370-9d6f-f1f2cd40d57d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:52:14.972174 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90218cd5-cbe2-4370-9d6f-f1f2cd40d57d None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:14.972605 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90218cd5-cbe2-4370-9d6f-f1f2cd40d57d None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:14.972933 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90218cd5-cbe2-4370-9d6f-f1f2cd40d57d None admin] PATH_INFO: `/v3/projects/service` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:14.982920 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90218cd5-cbe2-4370-9d6f-f1f2cd40d57d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:52:14.984685 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90218cd5-cbe2-4370-9d6f-f1f2cd40d57d None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:52:15.000781 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application [None req-90218cd5-cbe2-4370-9d6f-f1f2cd40d57d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 10:52:15.000781 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 10:52:15.000781 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 10:52:15.000781 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 10:52:15.000781 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 10:52:15.000781 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 10:52:15.000781 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 10:52:15.000781 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 10:52:15.000781 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 10:52:15.000781 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 10:52:15.000781 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 10:52:15.000781 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 10:52:15.000781 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 26 10:52:15.000781 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 26 10:52:15.000781 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 26 10:52:15.002011 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 26 10:52:15.002011 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 10:52:15.002011 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 10:52:15.002011 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 10:52:15.002011 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 10:52:15.002011 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 10:52:15.002011 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 10:52:15.002011 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 10:52:15.002011 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application with Lock( Aug 26 10:52:15.002011 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 10:52:15.002011 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._enter() Aug 26 10:52:15.002011 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 10:52:15.002011 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 10:52:15.002011 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 10:52:15.002011 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.creator() Aug 26 10:52:15.002011 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 10:52:15.002011 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application created_value = creator( Aug 26 10:52:15.003123 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 26 10:52:15.003123 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 26 10:52:15.003123 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 26 10:52:15.003123 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 26 10:52:15.003123 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 26 10:52:15.003123 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 26 10:52:15.003123 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 10:52:15.003123 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Aug 26 10:52:15.003123 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 299/599] 10.222.0.40 () {56 vars in 1111 bytes} [Fri Aug 26 10:52:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 10:52:15.010009 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe487c32-9a4f-4d3b-a9d3-045cbcb94068 None None] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:15.010009 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe487c32-9a4f-4d3b-a9d3-045cbcb94068 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:15.010009 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe487c32-9a4f-4d3b-a9d3-045cbcb94068 None None] PATH_INFO: `/` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:15.010622 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 300/600] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 26 10:52:15 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 10:52:15.011172 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe487c32-9a4f-4d3b-a9d3-045cbcb94068 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:52:15.027931 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c10ad2-742f-4a66-87e8-5509c5482303 None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:52:15.050182 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:52:15.073676 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f68fa9-2fe8-4af0-8e6e-45898d00c6c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:52:15.074332 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f68fa9-2fe8-4af0-8e6e-45898d00c6c2 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:15.074613 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f68fa9-2fe8-4af0-8e6e-45898d00c6c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:15.074893 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f68fa9-2fe8-4af0-8e6e-45898d00c6c2 None admin] PATH_INFO: `/v3/users` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:15.075635 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f68fa9-2fe8-4af0-8e6e-45898d00c6c2 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 10:52:15.075937 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f68fa9-2fe8-4af0-8e6e-45898d00c6c2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:52:15.077378 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f68fa9-2fe8-4af0-8e6e-45898d00c6c2 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:52:15.097376 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 301/601] 10.222.0.40 () {56 vars in 1139 bytes} [Fri Aug 26 10:52:14 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 10:52:15.098752 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed49c36e-6333-4199-b6b2-77cafc91d111 None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:15.099096 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed49c36e-6333-4199-b6b2-77cafc91d111 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:15.099315 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed49c36e-6333-4199-b6b2-77cafc91d111 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:15.118608 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-ed49c36e-6333-4199-b6b2-77cafc91d111 None None] MFA Rules not processed for user `ca155a48e2024c32af3fd2e19e5fac11`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 10:52:15.119808 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-ed49c36e-6333-4199-b6b2-77cafc91d111 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 10:52:15.138376 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:52:15.150496 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe487c32-9a4f-4d3b-a9d3-045cbcb94068 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:52:15.188037 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ff40a1-dd84-41ec-b7e8-7bc61758f80c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:52:15.189326 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ff40a1-dd84-41ec-b7e8-7bc61758f80c None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:15.189667 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ff40a1-dd84-41ec-b7e8-7bc61758f80c None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:15.190017 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ff40a1-dd84-41ec-b7e8-7bc61758f80c None admin] PATH_INFO: `/v3/projects` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:15.190790 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ff40a1-dd84-41ec-b7e8-7bc61758f80c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 10:52:15.191165 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ff40a1-dd84-41ec-b7e8-7bc61758f80c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:52:15.192815 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ff40a1-dd84-41ec-b7e8-7bc61758f80c None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:52:15.199770 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 302/602] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 26 10:52:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 10:52:15.200683 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed49c36e-6333-4199-b6b2-77cafc91d111 None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:52:15.212055 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 301/603] 10.222.0.40 () {56 vars in 1148 bytes} [Fri Aug 26 10:52:15 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 10:52:15.218175 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ff40a1-dd84-41ec-b7e8-7bc61758f80c None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:52:15.305050 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed49c36e-6333-4199-b6b2-77cafc91d111 None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:52:15.338362 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:52:15.354556 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e12e41-98b1-4583-a197-54c2fbc83ddc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:52:15.355197 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e12e41-98b1-4583-a197-54c2fbc83ddc None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:15.355479 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e12e41-98b1-4583-a197-54c2fbc83ddc None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:15.355761 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e12e41-98b1-4583-a197-54c2fbc83ddc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:15.373636 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ff40a1-dd84-41ec-b7e8-7bc61758f80c None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:52:15.375298 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e12e41-98b1-4583-a197-54c2fbc83ddc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:52:15.377467 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e12e41-98b1-4583-a197-54c2fbc83ddc None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:52:15.398187 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application [None req-c1e12e41-98b1-4583-a197-54c2fbc83ddc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 10:52:15.398187 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 10:52:15.398187 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 10:52:15.398187 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 10:52:15.398187 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 10:52:15.398187 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 10:52:15.398187 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 10:52:15.398187 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 10:52:15.398187 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 10:52:15.398187 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 10:52:15.398187 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 10:52:15.398187 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 10:52:15.398187 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 10:52:15.398187 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 10:52:15.398187 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 10:52:15.398187 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 10:52:15.399162 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 10:52:15.399162 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 10:52:15.399162 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 10:52:15.399162 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 10:52:15.399162 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 10:52:15.399162 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 10:52:15.399162 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 10:52:15.399162 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application with Lock( Aug 26 10:52:15.399162 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 10:52:15.399162 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._enter() Aug 26 10:52:15.399162 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 10:52:15.399162 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 10:52:15.399162 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 10:52:15.399162 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.creator() Aug 26 10:52:15.399162 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 10:52:15.399162 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application created_value = creator( Aug 26 10:52:15.399162 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 10:52:15.400180 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 10:52:15.400180 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 10:52:15.400180 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Aug 26 10:52:15.400180 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 303/604] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Aug 26 10:52:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 10:52:15.405316 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e12e41-98b1-4583-a197-54c2fbc83ddc None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:52:15.418321 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9276e6-b533-41cc-86e7-841c8a06e3b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:52:15.418987 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9276e6-b533-41cc-86e7-841c8a06e3b9 None admin] REQUEST_METHOD: `PUT` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:15.419267 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9276e6-b533-41cc-86e7-841c8a06e3b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:15.419542 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9276e6-b533-41cc-86e7-841c8a06e3b9 None admin] PATH_INFO: `/v3/projects/0e30ccf0c1204727b786876c141dbcd2/users/c277338a7f814cc6b01c5419775031bf/roles/3bda770fcd5c46cdbfabcd7404a25e66` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:15.441951 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a9276e6-b533-41cc-86e7-841c8a06e3b9 None admin] RBAC: Authorizing `identity:create_grant(project_id=0e30ccf0c1204727b786876c141dbcd2, user_id=c277338a7f814cc6b01c5419775031bf, role_id=3bda770fcd5c46cdbfabcd7404a25e66)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:52:15.443941 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a9276e6-b533-41cc-86e7-841c8a06e3b9 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:52:15.469506 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 302/605] 10.222.0.40 () {58 vars in 1439 bytes} [Fri Aug 26 10:52:15 2022] PUT /identity/v3/projects/0e30ccf0c1204727b786876c141dbcd2/users/c277338a7f814cc6b01c5419775031bf/roles/3bda770fcd5c46cdbfabcd7404a25e66 => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 10:52:15.548787 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e12e41-98b1-4583-a197-54c2fbc83ddc None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:52:15.591243 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0cb4be-c495-4a21-82ea-bb76c32bc8e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:52:15.591920 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0cb4be-c495-4a21-82ea-bb76c32bc8e3 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:15.592209 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0cb4be-c495-4a21-82ea-bb76c32bc8e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:15.592481 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0cb4be-c495-4a21-82ea-bb76c32bc8e3 None admin] PATH_INFO: `/v3/domains` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:15.593143 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac0cb4be-c495-4a21-82ea-bb76c32bc8e3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 10:52:15.593447 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac0cb4be-c495-4a21-82ea-bb76c32bc8e3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:52:15.594966 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac0cb4be-c495-4a21-82ea-bb76c32bc8e3 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:52:15.608331 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 304/606] 10.222.0.40 () {56 vars in 1109 bytes} [Fri Aug 26 10:52:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 10:52:15.613874 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9276e6-b533-41cc-86e7-841c8a06e3b9 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:52:15.628062 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:52:15.639323 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0cb4be-c495-4a21-82ea-bb76c32bc8e3 None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:52:15.664499 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:52:15.769297 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9276e6-b533-41cc-86e7-841c8a06e3b9 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:52:15.777826 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0cb4be-c495-4a21-82ea-bb76c32bc8e3 None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:52:15.806047 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf049d4-56d8-423a-a754-87d8c4d79ce3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:52:15.806636 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf049d4-56d8-423a-a754-87d8c4d79ce3 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:15.806900 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf049d4-56d8-423a-a754-87d8c4d79ce3 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:15.807171 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf049d4-56d8-423a-a754-87d8c4d79ce3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:15.814200 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf049d4-56d8-423a-a754-87d8c4d79ce3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:52:15.815837 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf049d4-56d8-423a-a754-87d8c4d79ce3 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:52:15.818218 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d318d6-992f-4f84-83f1-5b343999b0ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:52:15.819081 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d318d6-992f-4f84-83f1-5b343999b0ba None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:15.819652 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d318d6-992f-4f84-83f1-5b343999b0ba None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:15.820136 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d318d6-992f-4f84-83f1-5b343999b0ba None admin] PATH_INFO: `/v3/roles/service` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:15.827815 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application [None req-ecf049d4-56d8-423a-a754-87d8c4d79ce3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 10:52:15.827815 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 10:52:15.827815 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 10:52:15.827815 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 10:52:15.827815 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 10:52:15.827815 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 10:52:15.827815 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 10:52:15.827815 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 10:52:15.827815 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 10:52:15.827815 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 10:52:15.827815 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 10:52:15.827815 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 10:52:15.827815 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 26 10:52:15.827815 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 26 10:52:15.827815 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 26 10:52:15.828700 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 26 10:52:15.828700 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 10:52:15.828700 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 10:52:15.828700 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 10:52:15.828700 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 10:52:15.828700 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 10:52:15.828700 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 10:52:15.828700 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 10:52:15.828700 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application with Lock( Aug 26 10:52:15.828700 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 10:52:15.828700 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._enter() Aug 26 10:52:15.828700 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 10:52:15.828700 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 10:52:15.828700 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 10:52:15.828700 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.creator() Aug 26 10:52:15.828700 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 10:52:15.828700 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application created_value = creator( Aug 26 10:52:15.829958 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 26 10:52:15.829958 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 26 10:52:15.829958 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 26 10:52:15.829958 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 26 10:52:15.829958 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 26 10:52:15.829958 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 26 10:52:15.829958 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 10:52:15.829958 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Aug 26 10:52:15.829958 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 303/607] 10.222.0.40 () {56 vars in 1111 bytes} [Fri Aug 26 10:52:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 10:52:15.834744 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf049d4-56d8-423a-a754-87d8c4d79ce3 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:52:15.835163 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78d318d6-992f-4f84-83f1-5b343999b0ba None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 10:52:15.840210 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application [None req-78d318d6-992f-4f84-83f1-5b343999b0ba None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 26 10:52:15.840210 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 10:52:15.840210 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 10:52:15.840210 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 10:52:15.840210 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 10:52:15.840210 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 10:52:15.840210 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 10:52:15.840210 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 10:52:15.840210 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 10:52:15.840210 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 10:52:15.840210 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 10:52:15.840210 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 10:52:15.840210 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 10:52:15.840210 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 10:52:15.840210 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 10:52:15.840210 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 10:52:15.841128 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 10:52:15.841128 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 10:52:15.841128 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 10:52:15.841128 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 10:52:15.841128 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 10:52:15.841128 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 10:52:15.841128 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 10:52:15.841128 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 10:52:15.841128 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 10:52:15.841128 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 10:52:15.841128 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 10:52:15.841128 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application with Lock( Aug 26 10:52:15.841128 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 10:52:15.841128 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._enter() Aug 26 10:52:15.841128 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 10:52:15.841128 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 10:52:15.842094 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 10:52:15.842094 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.creator() Aug 26 10:52:15.842094 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 10:52:15.842094 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application created_value = creator( Aug 26 10:52:15.842094 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 10:52:15.842094 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 10:52:15.842094 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 10:52:15.842094 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 10:52:15.842094 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 10:52:15.842094 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 10:52:15.842094 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 26 10:52:15.842094 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Aug 26 10:52:15.844673 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 305/608] 10.222.0.40 () {56 vars in 1102 bytes} [Fri Aug 26 10:52:15 2022] GET /identity/v3/roles/service => generated 85 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 26 10:52:15.851128 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d318d6-992f-4f84-83f1-5b343999b0ba None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:52:15.904287 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:52:15.943814 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:52:15.948171 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf049d4-56d8-423a-a754-87d8c4d79ce3 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:52:15.968818 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d318d6-992f-4f84-83f1-5b343999b0ba None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:52:15.984841 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6838a5-bb27-4468-9676-8b699e358d87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:52:15.986296 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6838a5-bb27-4468-9676-8b699e358d87 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:15.986719 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6838a5-bb27-4468-9676-8b699e358d87 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:15.987427 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6838a5-bb27-4468-9676-8b699e358d87 None admin] PATH_INFO: `/v3/projects` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:15.988258 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf6838a5-bb27-4468-9676-8b699e358d87 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 10:52:15.988758 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf6838a5-bb27-4468-9676-8b699e358d87 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:52:15.990389 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf6838a5-bb27-4468-9676-8b699e358d87 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:52:16.004451 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 304/609] 10.222.0.40 () {56 vars in 1148 bytes} [Fri Aug 26 10:52:15 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 10:52:16.010517 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6838a5-bb27-4468-9676-8b699e358d87 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:52:16.012320 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf9bf01-6184-4d59-9b30-94e29f7b2277 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:52:16.013016 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf9bf01-6184-4d59-9b30-94e29f7b2277 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:16.013375 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf9bf01-6184-4d59-9b30-94e29f7b2277 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:16.013692 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf9bf01-6184-4d59-9b30-94e29f7b2277 None admin] PATH_INFO: `/v3/roles` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:16.014570 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf9bf01-6184-4d59-9b30-94e29f7b2277 None admin] RBAC: Adding query filter params (name=service) {{(pid=81423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 10:52:16.014970 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf9bf01-6184-4d59-9b30-94e29f7b2277 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:52:16.016533 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf9bf01-6184-4d59-9b30-94e29f7b2277 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:52:16.026363 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 306/610] 10.222.0.40 () {56 vars in 1103 bytes} [Fri Aug 26 10:52:15 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 10:52:16.032187 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf9bf01-6184-4d59-9b30-94e29f7b2277 None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:52:16.119155 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6838a5-bb27-4468-9676-8b699e358d87 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:52:16.144836 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:52:16.148048 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf9bf01-6184-4d59-9b30-94e29f7b2277 None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:52:16.157873 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44432cd1-6771-46db-9945-b9b86463eb32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:52:16.158918 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44432cd1-6771-46db-9945-b9b86463eb32 None admin] REQUEST_METHOD: `PUT` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:16.159374 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44432cd1-6771-46db-9945-b9b86463eb32 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:16.159835 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44432cd1-6771-46db-9945-b9b86463eb32 None admin] PATH_INFO: `/v3/projects/0e30ccf0c1204727b786876c141dbcd2/users/91da2acd2534423091537e1ba068d2b7/roles/3bda770fcd5c46cdbfabcd7404a25e66` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:16.176031 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44432cd1-6771-46db-9945-b9b86463eb32 None admin] RBAC: Authorizing `identity:create_grant(project_id=0e30ccf0c1204727b786876c141dbcd2, user_id=91da2acd2534423091537e1ba068d2b7, role_id=3bda770fcd5c46cdbfabcd7404a25e66)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:52:16.177715 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44432cd1-6771-46db-9945-b9b86463eb32 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:52:16.188902 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589382a4-db6e-4717-aa25-437031751eed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:52:16.189857 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589382a4-db6e-4717-aa25-437031751eed None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:16.190154 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589382a4-db6e-4717-aa25-437031751eed None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:16.190422 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589382a4-db6e-4717-aa25-437031751eed None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:16.193735 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 305/611] 10.222.0.40 () {58 vars in 1439 bytes} [Fri Aug 26 10:52:16 2022] PUT /identity/v3/projects/0e30ccf0c1204727b786876c141dbcd2/users/91da2acd2534423091537e1ba068d2b7/roles/3bda770fcd5c46cdbfabcd7404a25e66 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 10:52:16.200109 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:52:16.203165 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-589382a4-db6e-4717-aa25-437031751eed None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:52:16.205803 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-589382a4-db6e-4717-aa25-437031751eed None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:52:16.224412 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application [None req-589382a4-db6e-4717-aa25-437031751eed None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 10:52:16.224412 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 10:52:16.224412 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 10:52:16.224412 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 10:52:16.224412 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 10:52:16.224412 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 10:52:16.224412 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 10:52:16.224412 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 10:52:16.224412 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 10:52:16.224412 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 10:52:16.224412 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 10:52:16.224412 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 10:52:16.224412 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 10:52:16.224412 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 10:52:16.224412 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 10:52:16.224412 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 10:52:16.225648 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 10:52:16.225648 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 10:52:16.225648 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 10:52:16.225648 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 10:52:16.225648 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 10:52:16.225648 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 10:52:16.225648 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 10:52:16.225648 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application with Lock( Aug 26 10:52:16.225648 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 10:52:16.225648 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._enter() Aug 26 10:52:16.225648 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 10:52:16.225648 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 10:52:16.225648 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 10:52:16.225648 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.creator() Aug 26 10:52:16.225648 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 10:52:16.225648 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application created_value = creator( Aug 26 10:52:16.225648 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 10:52:16.226895 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 10:52:16.226895 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 10:52:16.226895 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Aug 26 10:52:16.228685 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 307/612] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Aug 26 10:52:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 10:52:16.234803 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44432cd1-6771-46db-9945-b9b86463eb32 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:52:16.371233 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44432cd1-6771-46db-9945-b9b86463eb32 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:52:16.409646 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe618d94-ea63-4e4e-9212-67cdbe1654e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:52:16.410517 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe618d94-ea63-4e4e-9212-67cdbe1654e8 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:16.410936 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe618d94-ea63-4e4e-9212-67cdbe1654e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:16.411343 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe618d94-ea63-4e4e-9212-67cdbe1654e8 None admin] PATH_INFO: `/v3/domains` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:16.412151 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe618d94-ea63-4e4e-9212-67cdbe1654e8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 10:52:16.412591 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe618d94-ea63-4e4e-9212-67cdbe1654e8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:52:16.414151 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe618d94-ea63-4e4e-9212-67cdbe1654e8 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:52:16.430068 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 306/613] 10.222.0.40 () {56 vars in 1109 bytes} [Fri Aug 26 10:52:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 10:52:16.435857 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589382a4-db6e-4717-aa25-437031751eed None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:52:16.567385 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589382a4-db6e-4717-aa25-437031751eed None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:52:16.609310 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859794af-f67b-4c59-b26d-9c3ed58206fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:52:16.610216 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859794af-f67b-4c59-b26d-9c3ed58206fa None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:16.611297 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859794af-f67b-4c59-b26d-9c3ed58206fa None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:16.611972 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859794af-f67b-4c59-b26d-9c3ed58206fa None admin] PATH_INFO: `/v3/users/swift` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:16.620273 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-859794af-f67b-4c59-b26d-9c3ed58206fa None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:52:16.621888 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-859794af-f67b-4c59-b26d-9c3ed58206fa None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:52:16.634235 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application [None req-859794af-f67b-4c59-b26d-9c3ed58206fa None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 26 10:52:16.634235 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 10:52:16.634235 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 10:52:16.634235 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 10:52:16.634235 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 10:52:16.634235 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 10:52:16.634235 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 10:52:16.634235 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 10:52:16.634235 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 10:52:16.634235 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 10:52:16.634235 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 10:52:16.634235 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 10:52:16.634235 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 26 10:52:16.634235 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 26 10:52:16.634235 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 26 10:52:16.634235 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 26 10:52:16.635380 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 10:52:16.635380 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 10:52:16.635380 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 26 10:52:16.635380 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 10:52:16.635380 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 26 10:52:16.635380 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 10:52:16.635380 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 10:52:16.635380 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 10:52:16.635380 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 10:52:16.635380 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 10:52:16.635380 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 10:52:16.635380 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application with Lock( Aug 26 10:52:16.635380 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 10:52:16.635380 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._enter() Aug 26 10:52:16.635380 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 10:52:16.635380 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 10:52:16.635380 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 10:52:16.635380 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.creator() Aug 26 10:52:16.636523 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 10:52:16.636523 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application created_value = creator( Aug 26 10:52:16.636523 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 26 10:52:16.636523 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 26 10:52:16.636523 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 26 10:52:16.636523 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 26 10:52:16.636523 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 26 10:52:16.636523 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 26 10:52:16.636523 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 26 10:52:16.636523 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Aug 26 10:52:16.638697 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 308/614] 10.222.0.40 () {56 vars in 1096 bytes} [Fri Aug 26 10:52:16 2022] GET /identity/v3/users/swift => generated 83 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 10:52:16.646956 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe618d94-ea63-4e4e-9212-67cdbe1654e8 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:52:16.710046 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:52:16.795542 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe618d94-ea63-4e4e-9212-67cdbe1654e8 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:52:16.848770 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7e4f33-a07d-4c24-9e47-5f8f92386098 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:52:16.849387 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7e4f33-a07d-4c24-9e47-5f8f92386098 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:16.849662 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7e4f33-a07d-4c24-9e47-5f8f92386098 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:16.849927 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7e4f33-a07d-4c24-9e47-5f8f92386098 None admin] PATH_INFO: `/v3/users` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:16.850692 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e7e4f33-a07d-4c24-9e47-5f8f92386098 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 10:52:16.850992 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e7e4f33-a07d-4c24-9e47-5f8f92386098 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:52:16.852412 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e7e4f33-a07d-4c24-9e47-5f8f92386098 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:52:16.884220 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 307/615] 10.222.0.40 () {56 vars in 1135 bytes} [Fri Aug 26 10:52:16 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 10:52:16.890006 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859794af-f67b-4c59-b26d-9c3ed58206fa None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:52:16.978177 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:52:16.984315 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751fb466-c7ca-4539-b921-a34fcfb222cb None None] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:16.984778 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751fb466-c7ca-4539-b921-a34fcfb222cb None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:16.984992 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751fb466-c7ca-4539-b921-a34fcfb222cb None None] PATH_INFO: `/` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:16.986665 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 308/616] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 26 10:52:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 26 10:52:16.993619 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8886b7-1a7e-4bcf-9d28-fb14b4ed9bdb None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:16.994199 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8886b7-1a7e-4bcf-9d28-fb14b4ed9bdb None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:16.994937 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8886b7-1a7e-4bcf-9d28-fb14b4ed9bdb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:17.017530 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-6d8886b7-1a7e-4bcf-9d28-fb14b4ed9bdb None None] MFA Rules not processed for user `ca155a48e2024c32af3fd2e19e5fac11`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 10:52:17.019576 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-6d8886b7-1a7e-4bcf-9d28-fb14b4ed9bdb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 10:52:17.060997 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859794af-f67b-4c59-b26d-9c3ed58206fa None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:52:17.129093 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb2d108-0bf6-47d4-9b26-ceaee2eb9e57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:52:17.129735 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb2d108-0bf6-47d4-9b26-ceaee2eb9e57 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:17.130026 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb2d108-0bf6-47d4-9b26-ceaee2eb9e57 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:17.130301 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb2d108-0bf6-47d4-9b26-ceaee2eb9e57 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:17.140854 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 309/617] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 26 10:52:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 26 10:52:17.143274 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9b4251-bca0-41d0-ba27-33b38ccc49bc None None] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:17.143559 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9b4251-bca0-41d0-ba27-33b38ccc49bc None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:17.143826 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9b4251-bca0-41d0-ba27-33b38ccc49bc None None] PATH_INFO: `/` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:17.144265 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb2d108-0bf6-47d4-9b26-ceaee2eb9e57 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:52:17.145356 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 310/618] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 26 10:52:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 10:52:17.145798 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb2d108-0bf6-47d4-9b26-ceaee2eb9e57 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:52:17.151114 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3487e31-79fa-4242-8f97-c799c1697e01 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:17.151494 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3487e31-79fa-4242-8f97-c799c1697e01 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:17.152126 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3487e31-79fa-4242-8f97-c799c1697e01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:17.167323 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application [None req-0eb2d108-0bf6-47d4-9b26-ceaee2eb9e57 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 10:52:17.167323 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 10:52:17.167323 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 10:52:17.167323 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 10:52:17.167323 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 10:52:17.167323 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 10:52:17.167323 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 10:52:17.167323 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 10:52:17.167323 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 10:52:17.167323 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 10:52:17.167323 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 10:52:17.167323 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 10:52:17.167323 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 10:52:17.167323 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 10:52:17.167323 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 10:52:17.167323 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 10:52:17.168339 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 10:52:17.168339 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 10:52:17.168339 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 10:52:17.168339 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 10:52:17.168339 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 10:52:17.168339 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 10:52:17.168339 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 10:52:17.168339 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application with Lock( Aug 26 10:52:17.168339 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 10:52:17.168339 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._enter() Aug 26 10:52:17.168339 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 10:52:17.168339 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 10:52:17.168339 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 10:52:17.168339 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.creator() Aug 26 10:52:17.168339 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 10:52:17.168339 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application created_value = creator( Aug 26 10:52:17.168339 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 10:52:17.169300 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 10:52:17.169300 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 10:52:17.169300 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Aug 26 10:52:17.169300 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 309/619] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Aug 26 10:52:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 279 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 10:52:17.174860 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb2d108-0bf6-47d4-9b26-ceaee2eb9e57 None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:52:17.187011 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-c3487e31-79fa-4242-8f97-c799c1697e01 None None] MFA Rules not processed for user `ca155a48e2024c32af3fd2e19e5fac11`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 10:52:17.188853 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-c3487e31-79fa-4242-8f97-c799c1697e01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 10:52:17.224098 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:52:17.321947 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb2d108-0bf6-47d4-9b26-ceaee2eb9e57 None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:52:17.326856 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:52:17.336801 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 311/620] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 26 10:52:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 26 10:52:17.379442 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6fe28ac-dac5-40ce-bf1d-38a7c5d7007f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:52:17.380058 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6fe28ac-dac5-40ce-bf1d-38a7c5d7007f None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:17.380382 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6fe28ac-dac5-40ce-bf1d-38a7c5d7007f None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:17.380655 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6fe28ac-dac5-40ce-bf1d-38a7c5d7007f None admin] PATH_INFO: `/v3/domains` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:17.381287 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6fe28ac-dac5-40ce-bf1d-38a7c5d7007f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 10:52:17.381585 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6fe28ac-dac5-40ce-bf1d-38a7c5d7007f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:52:17.382962 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6fe28ac-dac5-40ce-bf1d-38a7c5d7007f None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:52:17.401321 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 310/621] 10.222.0.40 () {56 vars in 1109 bytes} [Fri Aug 26 10:52:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 10:52:17.406891 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3487e31-79fa-4242-8f97-c799c1697e01 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:52:17.456836 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6fe28ac-dac5-40ce-bf1d-38a7c5d7007f None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:52:17.567148 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3487e31-79fa-4242-8f97-c799c1697e01 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:52:17.613796 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ceb9d92-05a4-4d89-919b-432125f8abdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:52:17.614806 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ceb9d92-05a4-4d89-919b-432125f8abdd None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:17.615289 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ceb9d92-05a4-4d89-919b-432125f8abdd None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:17.615783 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ceb9d92-05a4-4d89-919b-432125f8abdd None admin] PATH_INFO: `/v3/projects/service` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:17.626605 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ceb9d92-05a4-4d89-919b-432125f8abdd None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:52:17.628515 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ceb9d92-05a4-4d89-919b-432125f8abdd None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:52:17.638011 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6fe28ac-dac5-40ce-bf1d-38a7c5d7007f None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:52:17.651288 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application [None req-6ceb9d92-05a4-4d89-919b-432125f8abdd None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 10:52:17.651288 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 10:52:17.651288 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 10:52:17.651288 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 10:52:17.651288 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 10:52:17.651288 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 10:52:17.651288 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 10:52:17.651288 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 10:52:17.651288 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 10:52:17.651288 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 10:52:17.651288 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 10:52:17.651288 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 10:52:17.651288 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 26 10:52:17.651288 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 26 10:52:17.651288 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 26 10:52:17.652252 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 26 10:52:17.652252 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 10:52:17.652252 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 10:52:17.652252 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 10:52:17.652252 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 10:52:17.652252 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 10:52:17.652252 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 10:52:17.652252 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 10:52:17.652252 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application with Lock( Aug 26 10:52:17.652252 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 10:52:17.652252 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._enter() Aug 26 10:52:17.652252 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 10:52:17.652252 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 10:52:17.652252 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 10:52:17.652252 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.creator() Aug 26 10:52:17.652252 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 10:52:17.652252 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application created_value = creator( Aug 26 10:52:17.653210 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 26 10:52:17.653210 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 26 10:52:17.653210 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 26 10:52:17.653210 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 26 10:52:17.653210 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 26 10:52:17.653210 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 26 10:52:17.653210 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 10:52:17.653210 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Aug 26 10:52:17.655604 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 312/622] 10.222.0.40 () {56 vars in 1111 bytes} [Fri Aug 26 10:52:17 2022] GET /identity/v3/projects/service => generated 88 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 10:52:17.661068 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ceb9d92-05a4-4d89-919b-432125f8abdd None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:52:17.677520 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:52:17.720260 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b504d6-cefe-4ebc-8170-756f44d5f347 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:52:17.720575 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b504d6-cefe-4ebc-8170-756f44d5f347 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:17.720856 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b504d6-cefe-4ebc-8170-756f44d5f347 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:17.721126 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b504d6-cefe-4ebc-8170-756f44d5f347 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:17.733322 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b504d6-cefe-4ebc-8170-756f44d5f347 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 10:52:17.739374 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application [None req-88b504d6-cefe-4ebc-8170-756f44d5f347 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 26 10:52:17.739374 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 10:52:17.739374 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 10:52:17.739374 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 10:52:17.739374 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 10:52:17.739374 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 10:52:17.739374 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 10:52:17.739374 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 10:52:17.739374 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 10:52:17.739374 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 10:52:17.739374 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 10:52:17.739374 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 10:52:17.739374 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 10:52:17.739374 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 10:52:17.739374 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 10:52:17.739374 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 10:52:17.741320 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 10:52:17.741320 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 10:52:17.741320 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 10:52:17.741320 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 10:52:17.741320 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 10:52:17.741320 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 10:52:17.741320 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 10:52:17.741320 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 10:52:17.741320 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 10:52:17.741320 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 10:52:17.741320 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 10:52:17.741320 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application with Lock( Aug 26 10:52:17.741320 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 10:52:17.741320 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._enter() Aug 26 10:52:17.741320 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 10:52:17.741320 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 10:52:17.742956 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 10:52:17.742956 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.creator() Aug 26 10:52:17.742956 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 10:52:17.742956 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application created_value = creator( Aug 26 10:52:17.742956 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 10:52:17.742956 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 10:52:17.742956 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 10:52:17.742956 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 10:52:17.742956 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 10:52:17.742956 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 10:52:17.742956 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 26 10:52:17.742956 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Aug 26 10:52:17.742956 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 311/623] 10.222.0.40 () {56 vars in 1102 bytes} [Fri Aug 26 10:52:17 2022] GET /identity/v3/roles/service => generated 85 bytes in 284 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 26 10:52:17.742956 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b504d6-cefe-4ebc-8170-756f44d5f347 None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:52:17.775579 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:52:17.789188 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ceb9d92-05a4-4d89-919b-432125f8abdd None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:52:17.830246 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea34d856-4743-42b2-a0d3-20735d78e077 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:52:17.832720 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea34d856-4743-42b2-a0d3-20735d78e077 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:17.833541 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea34d856-4743-42b2-a0d3-20735d78e077 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:17.834417 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea34d856-4743-42b2-a0d3-20735d78e077 None admin] PATH_INFO: `/v3/projects` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:17.835970 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea34d856-4743-42b2-a0d3-20735d78e077 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 10:52:17.836836 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea34d856-4743-42b2-a0d3-20735d78e077 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:52:17.840238 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea34d856-4743-42b2-a0d3-20735d78e077 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:52:17.862752 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 313/624] 10.222.0.40 () {56 vars in 1148 bytes} [Fri Aug 26 10:52:17 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 10:52:17.863976 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b504d6-cefe-4ebc-8170-756f44d5f347 None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:52:17.865426 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea34d856-4743-42b2-a0d3-20735d78e077 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:52:17.900753 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90675044-5fbe-4d97-9a72-89b359c95375 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:52:17.901381 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90675044-5fbe-4d97-9a72-89b359c95375 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:17.901657 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90675044-5fbe-4d97-9a72-89b359c95375 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:17.901925 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90675044-5fbe-4d97-9a72-89b359c95375 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:17.913504 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90675044-5fbe-4d97-9a72-89b359c95375 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 10:52:17.918586 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application [None req-90675044-5fbe-4d97-9a72-89b359c95375 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 26 10:52:17.918586 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 10:52:17.918586 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 10:52:17.918586 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 10:52:17.918586 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 10:52:17.918586 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 10:52:17.918586 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 10:52:17.918586 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 10:52:17.918586 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 10:52:17.918586 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 10:52:17.918586 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 10:52:17.918586 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 10:52:17.918586 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 10:52:17.918586 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 10:52:17.918586 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 10:52:17.918586 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 10:52:17.919528 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 10:52:17.919528 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 10:52:17.919528 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 10:52:17.919528 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 10:52:17.919528 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 10:52:17.919528 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 10:52:17.919528 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 10:52:17.919528 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 10:52:17.919528 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 10:52:17.919528 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 10:52:17.919528 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 10:52:17.919528 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application with Lock( Aug 26 10:52:17.919528 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 10:52:17.919528 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._enter() Aug 26 10:52:17.919528 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 10:52:17.919528 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 10:52:17.920341 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 10:52:17.920341 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.creator() Aug 26 10:52:17.920341 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 10:52:17.920341 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application created_value = creator( Aug 26 10:52:17.920341 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 10:52:17.920341 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 10:52:17.920341 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 10:52:17.920341 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 10:52:17.920341 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 10:52:17.920341 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 10:52:17.920341 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 26 10:52:17.920341 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Aug 26 10:52:17.920341 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 312/625] 10.222.0.40 () {56 vars in 1102 bytes} [Fri Aug 26 10:52:17 2022] GET /identity/v3/roles/service => generated 85 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 10:52:17.921019 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90675044-5fbe-4d97-9a72-89b359c95375 None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:52:17.961927 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:52:18.000601 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea34d856-4743-42b2-a0d3-20735d78e077 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:52:18.019098 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:52:18.039648 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90675044-5fbe-4d97-9a72-89b359c95375 None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:52:18.046147 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0435ad2d-cde2-4c29-b5af-9914bdf4a376 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:52:18.046758 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0435ad2d-cde2-4c29-b5af-9914bdf4a376 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:18.047036 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0435ad2d-cde2-4c29-b5af-9914bdf4a376 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:18.047304 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0435ad2d-cde2-4c29-b5af-9914bdf4a376 None admin] PATH_INFO: `/v3/roles` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:18.048148 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0435ad2d-cde2-4c29-b5af-9914bdf4a376 None admin] RBAC: Adding query filter params (name=service) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 10:52:18.048509 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0435ad2d-cde2-4c29-b5af-9914bdf4a376 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:52:18.049867 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0435ad2d-cde2-4c29-b5af-9914bdf4a376 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:52:18.061262 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 314/626] 10.222.0.40 () {56 vars in 1103 bytes} [Fri Aug 26 10:52:17 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 10:52:18.062178 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0435ad2d-cde2-4c29-b5af-9914bdf4a376 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:52:18.087602 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24954e0e-fe98-44e0-a616-8a7392379487 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:52:18.089073 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24954e0e-fe98-44e0-a616-8a7392379487 None admin] REQUEST_METHOD: `PUT` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:18.089596 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24954e0e-fe98-44e0-a616-8a7392379487 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:18.090609 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24954e0e-fe98-44e0-a616-8a7392379487 None admin] PATH_INFO: `/v3/projects/0e30ccf0c1204727b786876c141dbcd2/users/6529bea296c9482a973ced5fe9f2212f/roles/3bda770fcd5c46cdbfabcd7404a25e66` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:18.112171 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24954e0e-fe98-44e0-a616-8a7392379487 None admin] RBAC: Authorizing `identity:create_grant(project_id=0e30ccf0c1204727b786876c141dbcd2, user_id=6529bea296c9482a973ced5fe9f2212f, role_id=3bda770fcd5c46cdbfabcd7404a25e66)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:52:18.114708 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24954e0e-fe98-44e0-a616-8a7392379487 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:52:18.132208 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 313/627] 10.222.0.40 () {58 vars in 1439 bytes} [Fri Aug 26 10:52:17 2022] PUT /identity/v3/projects/0e30ccf0c1204727b786876c141dbcd2/users/6529bea296c9482a973ced5fe9f2212f/roles/3bda770fcd5c46cdbfabcd7404a25e66 => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 10:52:18.133329 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24954e0e-fe98-44e0-a616-8a7392379487 None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:52:18.205285 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0435ad2d-cde2-4c29-b5af-9914bdf4a376 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:52:18.259923 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a441be86-c938-41be-b992-d2768c648861 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:52:18.260835 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a441be86-c938-41be-b992-d2768c648861 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:18.261418 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a441be86-c938-41be-b992-d2768c648861 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:18.261840 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a441be86-c938-41be-b992-d2768c648861 None admin] PATH_INFO: `/v3/roles` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:18.263071 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a441be86-c938-41be-b992-d2768c648861 None admin] RBAC: Adding query filter params (name=service) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 10:52:18.263543 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a441be86-c938-41be-b992-d2768c648861 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:52:18.265581 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a441be86-c938-41be-b992-d2768c648861 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:52:18.268245 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:52:18.273494 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24954e0e-fe98-44e0-a616-8a7392379487 None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:52:18.279454 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 315/628] 10.222.0.40 () {56 vars in 1103 bytes} [Fri Aug 26 10:52:18 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 10:52:18.281091 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57450ba5-4783-4c08-82ff-0057001c1f4f None None] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:18.281374 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57450ba5-4783-4c08-82ff-0057001c1f4f None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:18.281644 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57450ba5-4783-4c08-82ff-0057001c1f4f None None] PATH_INFO: `/` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:18.286246 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 316/629] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 26 10:52:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 10:52:18.289078 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4101ea11-6141-437d-85c2-e00ca0d58cd0 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:18.289078 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4101ea11-6141-437d-85c2-e00ca0d58cd0 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:18.289405 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4101ea11-6141-437d-85c2-e00ca0d58cd0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:18.308198 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-4101ea11-6141-437d-85c2-e00ca0d58cd0 None None] MFA Rules not processed for user `ca155a48e2024c32af3fd2e19e5fac11`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 10:52:18.309390 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-4101ea11-6141-437d-85c2-e00ca0d58cd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 10:52:18.321201 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23fbd054-6b43-4a59-b057-7af1cc963765 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:52:18.321830 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23fbd054-6b43-4a59-b057-7af1cc963765 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:18.322155 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23fbd054-6b43-4a59-b057-7af1cc963765 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:18.322422 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23fbd054-6b43-4a59-b057-7af1cc963765 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:18.324345 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:52:18.337733 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23fbd054-6b43-4a59-b057-7af1cc963765 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:52:18.339269 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23fbd054-6b43-4a59-b057-7af1cc963765 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:52:18.359311 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application [None req-23fbd054-6b43-4a59-b057-7af1cc963765 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 10:52:18.359311 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 10:52:18.359311 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 10:52:18.359311 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 10:52:18.359311 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 10:52:18.359311 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 10:52:18.359311 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 10:52:18.359311 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 10:52:18.359311 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 10:52:18.359311 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 10:52:18.359311 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 10:52:18.359311 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 10:52:18.359311 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 10:52:18.359311 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 10:52:18.359311 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 10:52:18.359311 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 10:52:18.360363 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 10:52:18.360363 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 10:52:18.360363 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 10:52:18.360363 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 10:52:18.360363 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 10:52:18.360363 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 10:52:18.360363 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 10:52:18.360363 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application with Lock( Aug 26 10:52:18.360363 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 10:52:18.360363 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._enter() Aug 26 10:52:18.360363 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 10:52:18.360363 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 10:52:18.360363 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 10:52:18.360363 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.creator() Aug 26 10:52:18.360363 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 10:52:18.360363 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application created_value = creator( Aug 26 10:52:18.360363 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 10:52:18.361454 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 10:52:18.361454 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 10:52:18.361454 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Aug 26 10:52:18.361454 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 314/630] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Aug 26 10:52:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 10:52:18.361706 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23fbd054-6b43-4a59-b057-7af1cc963765 None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:52:18.410001 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 317/631] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 26 10:52:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 26 10:52:18.410001 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4101ea11-6141-437d-85c2-e00ca0d58cd0 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:52:18.516779 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4101ea11-6141-437d-85c2-e00ca0d58cd0 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:52:18.536791 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23fbd054-6b43-4a59-b057-7af1cc963765 None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:52:18.544365 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:52:18.558217 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c34b9c-818f-468b-8f53-37a7cf43ca6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:52:18.558991 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c34b9c-818f-468b-8f53-37a7cf43ca6c None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:18.559487 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c34b9c-818f-468b-8f53-37a7cf43ca6c None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:18.559984 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c34b9c-818f-468b-8f53-37a7cf43ca6c None admin] PATH_INFO: `/v3/domains` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:18.562596 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c34b9c-818f-468b-8f53-37a7cf43ca6c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 10:52:18.562909 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c34b9c-818f-468b-8f53-37a7cf43ca6c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:52:18.565558 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c34b9c-818f-468b-8f53-37a7cf43ca6c None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:52:18.592804 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 318/632] 10.222.0.40 () {56 vars in 1109 bytes} [Fri Aug 26 10:52:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 10:52:18.600983 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c34b9c-818f-468b-8f53-37a7cf43ca6c None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:52:18.605913 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee98f6c-009d-407d-8a1b-c7ba563d858f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:52:18.606599 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee98f6c-009d-407d-8a1b-c7ba563d858f None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:18.606876 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee98f6c-009d-407d-8a1b-c7ba563d858f None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:18.607153 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee98f6c-009d-407d-8a1b-c7ba563d858f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:18.619283 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee98f6c-009d-407d-8a1b-c7ba563d858f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:52:18.620742 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee98f6c-009d-407d-8a1b-c7ba563d858f None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:52:18.635802 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application [None req-2ee98f6c-009d-407d-8a1b-c7ba563d858f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 10:52:18.635802 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 10:52:18.635802 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 10:52:18.635802 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 10:52:18.635802 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 10:52:18.635802 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 10:52:18.635802 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 10:52:18.635802 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 10:52:18.635802 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 10:52:18.635802 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 10:52:18.635802 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 10:52:18.635802 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 10:52:18.635802 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 10:52:18.635802 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 10:52:18.635802 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 10:52:18.635802 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 10:52:18.636851 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 10:52:18.636851 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 10:52:18.636851 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 10:52:18.636851 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 10:52:18.636851 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 10:52:18.636851 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 10:52:18.636851 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 10:52:18.636851 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application with Lock( Aug 26 10:52:18.636851 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 10:52:18.636851 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._enter() Aug 26 10:52:18.636851 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 10:52:18.636851 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 10:52:18.636851 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 10:52:18.636851 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.creator() Aug 26 10:52:18.636851 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 10:52:18.636851 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application created_value = creator( Aug 26 10:52:18.636851 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 10:52:18.638266 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 10:52:18.638266 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 10:52:18.638266 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Aug 26 10:52:18.638266 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 315/633] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Aug 26 10:52:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 276 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 10:52:18.644563 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee98f6c-009d-407d-8a1b-c7ba563d858f None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:52:18.691542 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:52:18.744532 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c34b9c-818f-468b-8f53-37a7cf43ca6c None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:52:18.757299 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee98f6c-009d-407d-8a1b-c7ba563d858f None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:52:18.784559 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e950438-99e1-498c-8fc5-5a7ef1e59666 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:52:18.785190 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e950438-99e1-498c-8fc5-5a7ef1e59666 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:18.785465 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e950438-99e1-498c-8fc5-5a7ef1e59666 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:18.785733 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e950438-99e1-498c-8fc5-5a7ef1e59666 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:18.793322 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e950438-99e1-498c-8fc5-5a7ef1e59666 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:52:18.794453 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b96c71d-71c8-44f7-8e24-3b387c769910 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:52:18.794769 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e950438-99e1-498c-8fc5-5a7ef1e59666 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:52:18.795612 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b96c71d-71c8-44f7-8e24-3b387c769910 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:18.796041 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b96c71d-71c8-44f7-8e24-3b387c769910 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:18.796475 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b96c71d-71c8-44f7-8e24-3b387c769910 None admin] PATH_INFO: `/v3/domains` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:18.797282 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b96c71d-71c8-44f7-8e24-3b387c769910 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 10:52:18.797576 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b96c71d-71c8-44f7-8e24-3b387c769910 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:52:18.799524 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b96c71d-71c8-44f7-8e24-3b387c769910 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:52:18.806535 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application [None req-9e950438-99e1-498c-8fc5-5a7ef1e59666 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 26 10:52:18.806535 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 10:52:18.806535 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 10:52:18.806535 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 10:52:18.806535 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 10:52:18.806535 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 10:52:18.806535 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 10:52:18.806535 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 10:52:18.806535 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 10:52:18.806535 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 10:52:18.806535 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 10:52:18.806535 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 10:52:18.806535 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 26 10:52:18.806535 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 26 10:52:18.806535 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 26 10:52:18.806535 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 26 10:52:18.807495 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 10:52:18.807495 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 10:52:18.807495 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 26 10:52:18.807495 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 10:52:18.807495 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 26 10:52:18.807495 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 10:52:18.807495 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 10:52:18.807495 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 10:52:18.807495 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 10:52:18.807495 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 10:52:18.807495 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 10:52:18.807495 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application with Lock( Aug 26 10:52:18.807495 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 10:52:18.807495 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._enter() Aug 26 10:52:18.807495 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 10:52:18.807495 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 10:52:18.807495 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 10:52:18.807495 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.creator() Aug 26 10:52:18.813466 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 10:52:18.813466 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application created_value = creator( Aug 26 10:52:18.813466 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 26 10:52:18.813466 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 26 10:52:18.813466 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 26 10:52:18.813466 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 26 10:52:18.813466 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 26 10:52:18.813466 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 26 10:52:18.813466 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 26 10:52:18.813466 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Aug 26 10:52:18.813466 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 319/634] 10.222.0.40 () {56 vars in 1093 bytes} [Fri Aug 26 10:52:18 2022] GET /identity/v3/users/nova => generated 82 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 10:52:18.813466 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f559eba-d35b-4268-a8c2-28e36c588de1 None None] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:18.813466 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f559eba-d35b-4268-a8c2-28e36c588de1 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:18.814552 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f559eba-d35b-4268-a8c2-28e36c588de1 None None] PATH_INFO: `/` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:18.814552 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 320/635] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 26 10:52:18 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 10:52:18.814552 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f559eba-d35b-4268-a8c2-28e36c588de1 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:52:18.819422 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 316/636] 10.222.0.40 () {56 vars in 1109 bytes} [Fri Aug 26 10:52:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 10:52:18.820432 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b96c71d-71c8-44f7-8e24-3b387c769910 None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:52:18.876502 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:52:18.929867 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:52:18.933691 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b96c71d-71c8-44f7-8e24-3b387c769910 None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:52:18.936235 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f559eba-d35b-4268-a8c2-28e36c588de1 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:52:18.969067 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e5887e-df7a-4561-abc4-8ef4e3d9893f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:52:18.969679 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e5887e-df7a-4561-abc4-8ef4e3d9893f None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:18.970595 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e5887e-df7a-4561-abc4-8ef4e3d9893f None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:18.970931 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e5887e-df7a-4561-abc4-8ef4e3d9893f None admin] PATH_INFO: `/v3/users` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:18.971316 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea46dab3-2465-4f8e-b36b-4256b89a525f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:52:18.971778 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e5887e-df7a-4561-abc4-8ef4e3d9893f None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 10:52:18.972073 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea46dab3-2465-4f8e-b36b-4256b89a525f None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:18.972180 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e5887e-df7a-4561-abc4-8ef4e3d9893f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:52:18.972483 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea46dab3-2465-4f8e-b36b-4256b89a525f None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:18.972829 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea46dab3-2465-4f8e-b36b-4256b89a525f None admin] PATH_INFO: `/v3/roles/service` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:18.973769 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e5887e-df7a-4561-abc4-8ef4e3d9893f None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:52:18.983616 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea46dab3-2465-4f8e-b36b-4256b89a525f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 10:52:18.989253 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application [None req-ea46dab3-2465-4f8e-b36b-4256b89a525f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 26 10:52:18.989253 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 10:52:18.989253 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 10:52:18.989253 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 10:52:18.989253 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 10:52:18.989253 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 10:52:18.989253 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 10:52:18.989253 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 10:52:18.989253 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 10:52:18.989253 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 10:52:18.989253 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 10:52:18.989253 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 10:52:18.989253 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 10:52:18.989253 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 10:52:18.989253 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 10:52:18.989253 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 10:52:18.995147 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 10:52:18.995147 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 10:52:18.995147 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 10:52:18.995147 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 10:52:18.995147 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 10:52:18.995147 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 10:52:18.995147 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 10:52:18.995147 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 10:52:18.995147 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 10:52:18.995147 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 10:52:18.995147 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 10:52:18.995147 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application with Lock( Aug 26 10:52:18.995147 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 10:52:18.995147 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._enter() Aug 26 10:52:18.995147 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 10:52:18.995147 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 10:52:18.996164 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 10:52:18.996164 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.creator() Aug 26 10:52:18.996164 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 10:52:18.996164 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application created_value = creator( Aug 26 10:52:18.996164 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 10:52:18.996164 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 10:52:18.996164 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 10:52:18.996164 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 10:52:18.996164 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 10:52:18.996164 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 10:52:18.996164 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 26 10:52:18.996164 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Aug 26 10:52:18.996164 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 321/637] 10.222.0.40 () {56 vars in 1102 bytes} [Fri Aug 26 10:52:18 2022] GET /identity/v3/roles/service => generated 85 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 10:52:18.996164 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93864717-7954-4728-a501-8368015fd966 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:18.997116 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93864717-7954-4728-a501-8368015fd966 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:18.997116 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93864717-7954-4728-a501-8368015fd966 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:19.002187 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 317/638] 10.222.0.40 () {56 vars in 1133 bytes} [Fri Aug 26 10:52:18 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 10:52:19.003126 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e5887e-df7a-4561-abc4-8ef4e3d9893f None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:52:19.011193 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-93864717-7954-4728-a501-8368015fd966 None None] MFA Rules not processed for user `ca155a48e2024c32af3fd2e19e5fac11`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 10:52:19.012363 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-93864717-7954-4728-a501-8368015fd966 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 10:52:19.080360 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 322/639] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 26 10:52:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 10:52:19.081426 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93864717-7954-4728-a501-8368015fd966 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:52:19.127587 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:52:19.131811 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e5887e-df7a-4561-abc4-8ef4e3d9893f None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:52:19.179107 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb12a78-43f1-4f06-bf0b-7192cf3db617 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:52:19.180001 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb12a78-43f1-4f06-bf0b-7192cf3db617 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:19.180867 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb12a78-43f1-4f06-bf0b-7192cf3db617 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:19.181292 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb12a78-43f1-4f06-bf0b-7192cf3db617 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:19.191322 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb12a78-43f1-4f06-bf0b-7192cf3db617 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:52:19.193710 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb12a78-43f1-4f06-bf0b-7192cf3db617 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:52:19.205756 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93864717-7954-4728-a501-8368015fd966 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:52:19.210381 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application [None req-3bb12a78-43f1-4f06-bf0b-7192cf3db617 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 26 10:52:19.210381 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 10:52:19.210381 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 10:52:19.210381 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 10:52:19.210381 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 10:52:19.210381 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 10:52:19.210381 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 10:52:19.210381 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 10:52:19.210381 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 10:52:19.210381 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 10:52:19.210381 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 10:52:19.210381 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 10:52:19.210381 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 26 10:52:19.210381 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 26 10:52:19.210381 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 26 10:52:19.210381 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 26 10:52:19.211491 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 10:52:19.211491 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 10:52:19.211491 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 26 10:52:19.211491 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 10:52:19.211491 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 26 10:52:19.211491 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 10:52:19.211491 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 10:52:19.211491 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 10:52:19.211491 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 10:52:19.211491 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 10:52:19.211491 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 10:52:19.211491 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application with Lock( Aug 26 10:52:19.211491 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 10:52:19.211491 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._enter() Aug 26 10:52:19.211491 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 10:52:19.211491 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 10:52:19.211491 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 10:52:19.211491 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.creator() Aug 26 10:52:19.212612 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 10:52:19.212612 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application created_value = creator( Aug 26 10:52:19.212612 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 26 10:52:19.212612 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 26 10:52:19.212612 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 26 10:52:19.212612 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 26 10:52:19.212612 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 26 10:52:19.212612 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 26 10:52:19.212612 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 26 10:52:19.212612 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Aug 26 10:52:19.215319 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 318/640] 10.222.0.40 () {56 vars in 1099 bytes} [Fri Aug 26 10:52:19 2022] GET /identity/v3/users/glance => generated 84 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 10:52:19.216804 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb12a78-43f1-4f06-bf0b-7192cf3db617 None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:52:19.241458 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:52:19.244267 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac8381b-42b6-4998-9f4d-a5109bc06667 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:52:19.244874 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac8381b-42b6-4998-9f4d-a5109bc06667 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:19.245195 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac8381b-42b6-4998-9f4d-a5109bc06667 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:19.245466 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac8381b-42b6-4998-9f4d-a5109bc06667 None admin] PATH_INFO: `/v3/roles` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:19.246320 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac8381b-42b6-4998-9f4d-a5109bc06667 None admin] RBAC: Adding query filter params (name=service) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 10:52:19.247704 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac8381b-42b6-4998-9f4d-a5109bc06667 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:52:19.249212 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac8381b-42b6-4998-9f4d-a5109bc06667 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:52:19.262142 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 323/641] 10.222.0.40 () {56 vars in 1103 bytes} [Fri Aug 26 10:52:19 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 10:52:19.263236 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac8381b-42b6-4998-9f4d-a5109bc06667 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:52:19.332052 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb12a78-43f1-4f06-bf0b-7192cf3db617 None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:52:19.367693 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:52:19.373651 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac8381b-42b6-4998-9f4d-a5109bc06667 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:52:19.384035 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde4b20a-9155-43d5-8a02-ec06e0f32686 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:52:19.384663 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde4b20a-9155-43d5-8a02-ec06e0f32686 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:19.384940 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde4b20a-9155-43d5-8a02-ec06e0f32686 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:19.385209 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde4b20a-9155-43d5-8a02-ec06e0f32686 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:19.413914 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500de3d6-95e0-40d6-9650-fd5380f6ba89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:52:19.414807 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500de3d6-95e0-40d6-9650-fd5380f6ba89 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:19.415082 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500de3d6-95e0-40d6-9650-fd5380f6ba89 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:19.415349 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500de3d6-95e0-40d6-9650-fd5380f6ba89 None admin] PATH_INFO: `/v3/users` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:19.416066 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-500de3d6-95e0-40d6-9650-fd5380f6ba89 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 10:52:19.416364 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-500de3d6-95e0-40d6-9650-fd5380f6ba89 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:52:19.417921 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-500de3d6-95e0-40d6-9650-fd5380f6ba89 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:52:19.436437 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 324/642] 10.222.0.40 () {56 vars in 1137 bytes} [Fri Aug 26 10:52:19 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 10:52:19.437165 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500de3d6-95e0-40d6-9650-fd5380f6ba89 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:52:19.468768 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde4b20a-9155-43d5-8a02-ec06e0f32686 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:52:19.470345 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde4b20a-9155-43d5-8a02-ec06e0f32686 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:52:19.488904 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:52:19.543422 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500de3d6-95e0-40d6-9650-fd5380f6ba89 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:52:19.583762 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 319/643] 10.222.0.40 () {58 vars in 1282 bytes} [Fri Aug 26 10:52:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 363 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 26 10:52:19.583959 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392aea34-ad13-423c-8621-39a3aa9c4e0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:52:19.584098 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde4b20a-9155-43d5-8a02-ec06e0f32686 None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:52:19.584194 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392aea34-ad13-423c-8621-39a3aa9c4e0f None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:19.584194 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392aea34-ad13-423c-8621-39a3aa9c4e0f None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:19.584194 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392aea34-ad13-423c-8621-39a3aa9c4e0f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:19.594128 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-392aea34-ad13-423c-8621-39a3aa9c4e0f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:52:19.596459 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-392aea34-ad13-423c-8621-39a3aa9c4e0f None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:52:19.612528 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application [None req-392aea34-ad13-423c-8621-39a3aa9c4e0f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 10:52:19.612528 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 10:52:19.612528 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 10:52:19.612528 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 10:52:19.612528 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 10:52:19.612528 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 10:52:19.612528 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 10:52:19.612528 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 10:52:19.612528 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 10:52:19.612528 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 10:52:19.612528 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 10:52:19.612528 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 10:52:19.612528 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 10:52:19.612528 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 10:52:19.612528 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 10:52:19.612528 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 10:52:19.613675 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 10:52:19.613675 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 10:52:19.613675 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 10:52:19.613675 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 10:52:19.613675 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 10:52:19.613675 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 10:52:19.613675 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 10:52:19.613675 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application with Lock( Aug 26 10:52:19.613675 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 10:52:19.613675 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._enter() Aug 26 10:52:19.613675 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 10:52:19.613675 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 10:52:19.613675 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 10:52:19.613675 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.creator() Aug 26 10:52:19.613675 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 10:52:19.613675 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application created_value = creator( Aug 26 10:52:19.613675 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 10:52:19.615197 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 10:52:19.615197 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 10:52:19.615197 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Aug 26 10:52:19.615197 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 325/644] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Aug 26 10:52:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 10:52:19.615197 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392aea34-ad13-423c-8621-39a3aa9c4e0f None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:52:19.627333 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:52:19.695248 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde4b20a-9155-43d5-8a02-ec06e0f32686 None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:52:19.711963 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:52:19.725229 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392aea34-ad13-423c-8621-39a3aa9c4e0f None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:52:19.731001 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5275a2de-13eb-429d-b4d1-ae4e74fae268 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:52:19.731632 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5275a2de-13eb-429d-b4d1-ae4e74fae268 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:19.731904 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5275a2de-13eb-429d-b4d1-ae4e74fae268 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:19.732453 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5275a2de-13eb-429d-b4d1-ae4e74fae268 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:19.743599 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5275a2de-13eb-429d-b4d1-ae4e74fae268 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 10:52:19.748337 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application [None req-5275a2de-13eb-429d-b4d1-ae4e74fae268 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 26 10:52:19.748337 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 10:52:19.748337 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 10:52:19.748337 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 10:52:19.748337 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 10:52:19.748337 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 10:52:19.748337 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 10:52:19.748337 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 10:52:19.748337 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 10:52:19.748337 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 10:52:19.748337 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 10:52:19.748337 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 10:52:19.748337 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 10:52:19.748337 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 10:52:19.748337 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 10:52:19.748337 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 10:52:19.749424 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 10:52:19.749424 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 10:52:19.749424 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 10:52:19.749424 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 10:52:19.749424 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 10:52:19.749424 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 10:52:19.749424 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 10:52:19.749424 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 10:52:19.749424 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 10:52:19.749424 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 10:52:19.749424 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 10:52:19.749424 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application with Lock( Aug 26 10:52:19.749424 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 10:52:19.749424 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._enter() Aug 26 10:52:19.749424 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 10:52:19.749424 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 10:52:19.750579 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 10:52:19.750579 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.creator() Aug 26 10:52:19.750579 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 10:52:19.750579 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application created_value = creator( Aug 26 10:52:19.750579 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 10:52:19.750579 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 10:52:19.750579 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 10:52:19.750579 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 10:52:19.750579 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 10:52:19.750579 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 10:52:19.750579 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 26 10:52:19.750579 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Aug 26 10:52:19.750579 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 320/645] 10.222.0.40 () {56 vars in 1102 bytes} [Fri Aug 26 10:52:19 2022] GET /identity/v3/roles/service => generated 85 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 10:52:19.750579 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5275a2de-13eb-429d-b4d1-ae4e74fae268 None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:52:19.767313 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c5cd5f-83fb-48ab-9877-0b738fc1a414 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:52:19.768200 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c5cd5f-83fb-48ab-9877-0b738fc1a414 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:19.768652 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c5cd5f-83fb-48ab-9877-0b738fc1a414 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:19.768924 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c5cd5f-83fb-48ab-9877-0b738fc1a414 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:19.841727 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c5cd5f-83fb-48ab-9877-0b738fc1a414 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:52:19.843139 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c5cd5f-83fb-48ab-9877-0b738fc1a414 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:52:19.856262 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5275a2de-13eb-429d-b4d1-ae4e74fae268 None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:52:19.862248 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:52:19.897205 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a100d8-3866-48f8-874d-67e65cbc91d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:52:19.897835 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a100d8-3866-48f8-874d-67e65cbc91d9 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:19.898142 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a100d8-3866-48f8-874d-67e65cbc91d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:19.898410 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a100d8-3866-48f8-874d-67e65cbc91d9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:19.909287 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a100d8-3866-48f8-874d-67e65cbc91d9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:52:19.910753 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a100d8-3866-48f8-874d-67e65cbc91d9 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:52:19.926107 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application [None req-57a100d8-3866-48f8-874d-67e65cbc91d9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 10:52:19.926107 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 10:52:19.926107 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 10:52:19.926107 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 10:52:19.926107 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 10:52:19.926107 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 10:52:19.926107 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 10:52:19.926107 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 10:52:19.926107 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 10:52:19.926107 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 10:52:19.926107 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 10:52:19.926107 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 10:52:19.926107 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 10:52:19.926107 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 10:52:19.926107 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 10:52:19.926107 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 10:52:19.927078 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 10:52:19.927078 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 10:52:19.927078 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 10:52:19.927078 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 10:52:19.927078 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 10:52:19.927078 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 10:52:19.927078 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 10:52:19.927078 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application with Lock( Aug 26 10:52:19.927078 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 10:52:19.927078 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._enter() Aug 26 10:52:19.927078 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 10:52:19.927078 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 10:52:19.927078 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 10:52:19.927078 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.creator() Aug 26 10:52:19.927078 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 10:52:19.927078 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application created_value = creator( Aug 26 10:52:19.927078 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 10:52:19.928558 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 10:52:19.928558 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 10:52:19.928558 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Aug 26 10:52:19.928558 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 321/646] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Aug 26 10:52:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 10:52:19.928558 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a100d8-3866-48f8-874d-67e65cbc91d9 None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:52:19.958828 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:52:19.962189 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 326/647] 10.222.0.40 () {58 vars in 1282 bytes} [Fri Aug 26 10:52:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 26 10:52:19.962954 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c5cd5f-83fb-48ab-9877-0b738fc1a414 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:52:20.052812 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a100d8-3866-48f8-874d-67e65cbc91d9 None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:52:20.065833 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c5cd5f-83fb-48ab-9877-0b738fc1a414 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:52:20.076180 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:52:20.090417 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712c0f86-f87f-4c44-99d5-fe474716a852 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:52:20.091307 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712c0f86-f87f-4c44-99d5-fe474716a852 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:20.091584 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712c0f86-f87f-4c44-99d5-fe474716a852 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:20.092062 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712c0f86-f87f-4c44-99d5-fe474716a852 None admin] PATH_INFO: `/v3/domains` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:20.093082 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-712c0f86-f87f-4c44-99d5-fe474716a852 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 10:52:20.093476 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-712c0f86-f87f-4c44-99d5-fe474716a852 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:52:20.095405 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-712c0f86-f87f-4c44-99d5-fe474716a852 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:52:20.104090 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e6bcf8-4d2e-44a9-bb4e-f71e50605bf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:52:20.104694 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e6bcf8-4d2e-44a9-bb4e-f71e50605bf7 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:20.104968 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e6bcf8-4d2e-44a9-bb4e-f71e50605bf7 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:20.105236 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e6bcf8-4d2e-44a9-bb4e-f71e50605bf7 None admin] PATH_INFO: `/v3/roles` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:20.105894 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e6bcf8-4d2e-44a9-bb4e-f71e50605bf7 None admin] RBAC: Adding query filter params (name=service) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 10:52:20.106222 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e6bcf8-4d2e-44a9-bb4e-f71e50605bf7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:52:20.112460 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 322/648] 10.222.0.40 () {56 vars in 1109 bytes} [Fri Aug 26 10:52:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 10:52:20.112614 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e6bcf8-4d2e-44a9-bb4e-f71e50605bf7 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:52:20.112705 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712c0f86-f87f-4c44-99d5-fe474716a852 None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:52:20.122784 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 327/649] 10.222.0.40 () {56 vars in 1103 bytes} [Fri Aug 26 10:52:19 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 10:52:20.124194 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e6bcf8-4d2e-44a9-bb4e-f71e50605bf7 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:52:20.203395 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:52:20.215933 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712c0f86-f87f-4c44-99d5-fe474716a852 None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:52:20.228950 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e6bcf8-4d2e-44a9-bb4e-f71e50605bf7 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:52:20.249997 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe1765b-0b11-40e9-8e56-5744301e8b16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:52:20.250604 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe1765b-0b11-40e9-8e56-5744301e8b16 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:20.250875 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe1765b-0b11-40e9-8e56-5744301e8b16 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:20.251170 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe1765b-0b11-40e9-8e56-5744301e8b16 None admin] PATH_INFO: `/v3/domains` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:20.251802 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe1765b-0b11-40e9-8e56-5744301e8b16 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 10:52:20.252095 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe1765b-0b11-40e9-8e56-5744301e8b16 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:52:20.253432 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe1765b-0b11-40e9-8e56-5744301e8b16 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:52:20.261787 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0daffe-054c-464b-b36f-2b19b1c56433 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:52:20.262436 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0daffe-054c-464b-b36f-2b19b1c56433 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:20.262714 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0daffe-054c-464b-b36f-2b19b1c56433 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:20.262988 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0daffe-054c-464b-b36f-2b19b1c56433 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:20.264422 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 323/650] 10.222.0.40 () {56 vars in 1109 bytes} [Fri Aug 26 10:52:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 10:52:20.265194 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe1765b-0b11-40e9-8e56-5744301e8b16 None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:52:20.274139 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d0daffe-054c-464b-b36f-2b19b1c56433 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:52:20.275601 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d0daffe-054c-464b-b36f-2b19b1c56433 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:52:20.290113 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application [None req-9d0daffe-054c-464b-b36f-2b19b1c56433 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 10:52:20.290113 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 10:52:20.290113 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 10:52:20.290113 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 10:52:20.290113 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 10:52:20.290113 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 10:52:20.290113 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 10:52:20.290113 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 10:52:20.290113 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 10:52:20.290113 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 10:52:20.290113 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 10:52:20.290113 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 10:52:20.290113 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 10:52:20.290113 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 10:52:20.290113 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 10:52:20.290113 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 10:52:20.291261 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 10:52:20.291261 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 10:52:20.291261 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 10:52:20.291261 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 10:52:20.291261 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 10:52:20.291261 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 10:52:20.291261 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 10:52:20.291261 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application with Lock( Aug 26 10:52:20.291261 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 10:52:20.291261 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._enter() Aug 26 10:52:20.291261 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 10:52:20.291261 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 10:52:20.291261 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 10:52:20.291261 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.creator() Aug 26 10:52:20.291261 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 10:52:20.291261 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application created_value = creator( Aug 26 10:52:20.291261 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 10:52:20.292159 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 10:52:20.292159 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 10:52:20.292159 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Aug 26 10:52:20.292159 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 328/651] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Aug 26 10:52:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 10:52:20.292159 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0daffe-054c-464b-b36f-2b19b1c56433 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:52:20.317890 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:52:20.371454 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe1765b-0b11-40e9-8e56-5744301e8b16 None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:52:20.392457 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0daffe-054c-464b-b36f-2b19b1c56433 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:52:20.404989 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a7d6d9-a36f-471b-8674-e94bdb7a248c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:52:20.405613 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a7d6d9-a36f-471b-8674-e94bdb7a248c None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:20.405888 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a7d6d9-a36f-471b-8674-e94bdb7a248c None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:20.406183 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a7d6d9-a36f-471b-8674-e94bdb7a248c None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:20.413531 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a7d6d9-a36f-471b-8674-e94bdb7a248c None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:52:20.414965 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a7d6d9-a36f-471b-8674-e94bdb7a248c None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:52:20.425038 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f88746-f7ce-4e4c-aeb0-de1494234cb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:52:20.425665 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f88746-f7ce-4e4c-aeb0-de1494234cb5 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:20.425946 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f88746-f7ce-4e4c-aeb0-de1494234cb5 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:20.426143 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application [None req-a6a7d6d9-a36f-471b-8674-e94bdb7a248c None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 26 10:52:20.426143 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 10:52:20.426143 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 10:52:20.426143 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 10:52:20.426143 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 10:52:20.426143 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 10:52:20.426143 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 10:52:20.426143 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 10:52:20.426143 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 10:52:20.426143 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 10:52:20.426143 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 10:52:20.426143 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 10:52:20.426143 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 26 10:52:20.426143 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 26 10:52:20.426143 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 26 10:52:20.426143 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 26 10:52:20.427101 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 10:52:20.427101 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 10:52:20.427101 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 26 10:52:20.427101 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 10:52:20.427101 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 26 10:52:20.427101 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 10:52:20.427101 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 10:52:20.427101 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 10:52:20.427101 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 10:52:20.427101 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 10:52:20.427101 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 10:52:20.427101 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application with Lock( Aug 26 10:52:20.427101 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 10:52:20.427101 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._enter() Aug 26 10:52:20.427101 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 10:52:20.427101 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 10:52:20.427101 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 10:52:20.427101 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.creator() Aug 26 10:52:20.428046 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 10:52:20.428046 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application created_value = creator( Aug 26 10:52:20.428046 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 26 10:52:20.428046 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 26 10:52:20.428046 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 26 10:52:20.428046 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 26 10:52:20.428046 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 26 10:52:20.428046 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 26 10:52:20.428046 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 26 10:52:20.428046 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Aug 26 10:52:20.428585 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f88746-f7ce-4e4c-aeb0-de1494234cb5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:20.428661 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 324/652] 10.222.0.40 () {56 vars in 1099 bytes} [Fri Aug 26 10:52:20 2022] GET /identity/v3/users/cinder => generated 84 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 10:52:20.428661 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d644e2db-1b26-4c88-b2fb-f94af34e6e14 None None] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:20.428977 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d644e2db-1b26-4c88-b2fb-f94af34e6e14 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:20.429213 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d644e2db-1b26-4c88-b2fb-f94af34e6e14 None None] PATH_INFO: `/` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:20.430471 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 325/653] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 26 10:52:20 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 10:52:20.431161 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d644e2db-1b26-4c88-b2fb-f94af34e6e14 None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:52:20.442850 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38f88746-f7ce-4e4c-aeb0-de1494234cb5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:52:20.443410 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:52:20.444321 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38f88746-f7ce-4e4c-aeb0-de1494234cb5 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:52:20.459301 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application [None req-38f88746-f7ce-4e4c-aeb0-de1494234cb5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 10:52:20.459301 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 10:52:20.459301 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 10:52:20.459301 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 10:52:20.459301 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 10:52:20.459301 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 10:52:20.459301 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 10:52:20.459301 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 10:52:20.459301 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 10:52:20.459301 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 10:52:20.459301 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 10:52:20.459301 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 10:52:20.459301 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 10:52:20.459301 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 10:52:20.459301 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 10:52:20.459301 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 10:52:20.460289 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 10:52:20.460289 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 10:52:20.460289 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 10:52:20.460289 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 10:52:20.460289 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 10:52:20.460289 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 10:52:20.460289 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 10:52:20.460289 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application with Lock( Aug 26 10:52:20.460289 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 10:52:20.460289 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._enter() Aug 26 10:52:20.460289 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 10:52:20.460289 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 10:52:20.460289 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 10:52:20.460289 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.creator() Aug 26 10:52:20.460289 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 10:52:20.460289 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application created_value = creator( Aug 26 10:52:20.460289 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 10:52:20.461187 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 10:52:20.461187 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 10:52:20.461187 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Aug 26 10:52:20.461187 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 329/654] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Aug 26 10:52:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 10:52:20.461187 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f88746-f7ce-4e4c-aeb0-de1494234cb5 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:52:20.534962 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:52:20.539944 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d644e2db-1b26-4c88-b2fb-f94af34e6e14 None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:52:20.559185 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f88746-f7ce-4e4c-aeb0-de1494234cb5 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:52:20.575268 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1059303a-e7b9-4008-b7bd-e21acc90b6e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:52:20.575887 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1059303a-e7b9-4008-b7bd-e21acc90b6e8 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:20.576168 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1059303a-e7b9-4008-b7bd-e21acc90b6e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:20.576436 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1059303a-e7b9-4008-b7bd-e21acc90b6e8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:20.583413 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1059303a-e7b9-4008-b7bd-e21acc90b6e8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:52:20.584829 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1059303a-e7b9-4008-b7bd-e21acc90b6e8 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:52:20.592660 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d61aa1-5fe8-4ce8-9699-3500e9b4fda9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:52:20.593264 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d61aa1-5fe8-4ce8-9699-3500e9b4fda9 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:20.593545 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d61aa1-5fe8-4ce8-9699-3500e9b4fda9 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:20.593832 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d61aa1-5fe8-4ce8-9699-3500e9b4fda9 None admin] PATH_INFO: `/v3/domains` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:20.594490 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d61aa1-5fe8-4ce8-9699-3500e9b4fda9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 10:52:20.595324 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d61aa1-5fe8-4ce8-9699-3500e9b4fda9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:52:20.596231 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application [None req-1059303a-e7b9-4008-b7bd-e21acc90b6e8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 10:52:20.596231 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 10:52:20.596231 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 10:52:20.596231 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 10:52:20.596231 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 10:52:20.596231 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 10:52:20.596231 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 10:52:20.596231 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 10:52:20.596231 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 10:52:20.596231 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 10:52:20.596231 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 10:52:20.596231 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 10:52:20.596231 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 26 10:52:20.596231 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 26 10:52:20.596231 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 26 10:52:20.597092 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 26 10:52:20.597092 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 10:52:20.597092 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 10:52:20.597092 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 10:52:20.597092 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 10:52:20.597092 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 10:52:20.597092 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 10:52:20.597092 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 10:52:20.597092 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application with Lock( Aug 26 10:52:20.597092 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 10:52:20.597092 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._enter() Aug 26 10:52:20.597092 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 10:52:20.597092 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 10:52:20.597092 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 10:52:20.597092 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.creator() Aug 26 10:52:20.597092 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 10:52:20.597092 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application created_value = creator( Aug 26 10:52:20.598271 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 26 10:52:20.598271 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 26 10:52:20.598271 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 26 10:52:20.598271 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 26 10:52:20.598271 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 26 10:52:20.598271 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 26 10:52:20.598271 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 10:52:20.598271 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Aug 26 10:52:20.598976 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d61aa1-5fe8-4ce8-9699-3500e9b4fda9 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:52:20.599065 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 326/655] 10.222.0.40 () {56 vars in 1111 bytes} [Fri Aug 26 10:52:20 2022] GET /identity/v3/projects/service => generated 88 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 10:52:20.599065 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1059303a-e7b9-4008-b7bd-e21acc90b6e8 None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:52:20.612870 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 330/656] 10.222.0.40 () {56 vars in 1109 bytes} [Fri Aug 26 10:52:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 10:52:20.614436 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b6c475-28c2-4a8f-bbd2-4d81336a1d38 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:20.614717 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b6c475-28c2-4a8f-bbd2-4d81336a1d38 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:20.614985 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b6c475-28c2-4a8f-bbd2-4d81336a1d38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:20.632587 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-08b6c475-28c2-4a8f-bbd2-4d81336a1d38 None None] MFA Rules not processed for user `ca155a48e2024c32af3fd2e19e5fac11`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 10:52:20.633713 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-08b6c475-28c2-4a8f-bbd2-4d81336a1d38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 10:52:20.672235 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:52:20.700563 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 331/657] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 26 10:52:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 10:52:20.701396 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b6c475-28c2-4a8f-bbd2-4d81336a1d38 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:52:20.713989 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1059303a-e7b9-4008-b7bd-e21acc90b6e8 None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:52:20.749444 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eecfb22-8ce1-4e72-828c-b5daa224c57b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:52:20.750081 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eecfb22-8ce1-4e72-828c-b5daa224c57b None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:20.750363 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eecfb22-8ce1-4e72-828c-b5daa224c57b None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:20.750633 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eecfb22-8ce1-4e72-828c-b5daa224c57b None admin] PATH_INFO: `/v3/users` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:20.752092 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eecfb22-8ce1-4e72-828c-b5daa224c57b None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 10:52:20.752292 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eecfb22-8ce1-4e72-828c-b5daa224c57b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:52:20.753709 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eecfb22-8ce1-4e72-828c-b5daa224c57b None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:52:20.770468 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 327/658] 10.222.0.40 () {56 vars in 1137 bytes} [Fri Aug 26 10:52:20 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 10:52:20.771263 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eecfb22-8ce1-4e72-828c-b5daa224c57b None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:52:20.776729 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:52:20.819302 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b6c475-28c2-4a8f-bbd2-4d81336a1d38 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:52:20.862101 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae9e96b-a985-456f-b216-6595d1cfc4b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:52:20.862716 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae9e96b-a985-456f-b216-6595d1cfc4b5 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:20.862990 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae9e96b-a985-456f-b216-6595d1cfc4b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:20.863257 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae9e96b-a985-456f-b216-6595d1cfc4b5 None admin] PATH_INFO: `/v3/domains` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:20.863946 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae9e96b-a985-456f-b216-6595d1cfc4b5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 10:52:20.864668 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae9e96b-a985-456f-b216-6595d1cfc4b5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:52:20.866162 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae9e96b-a985-456f-b216-6595d1cfc4b5 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:52:20.878621 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 332/659] 10.222.0.40 () {56 vars in 1109 bytes} [Fri Aug 26 10:52:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 10:52:20.879347 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae9e96b-a985-456f-b216-6595d1cfc4b5 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:52:20.883987 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eecfb22-8ce1-4e72-828c-b5daa224c57b None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:52:20.912280 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:52:20.921213 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eafc191-a7af-4458-b1fe-5110a8c52c3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:52:20.921825 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eafc191-a7af-4458-b1fe-5110a8c52c3e None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:20.922136 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eafc191-a7af-4458-b1fe-5110a8c52c3e None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:20.922410 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eafc191-a7af-4458-b1fe-5110a8c52c3e None admin] PATH_INFO: `/v3/projects` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:20.923657 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eafc191-a7af-4458-b1fe-5110a8c52c3e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 10:52:20.923977 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eafc191-a7af-4458-b1fe-5110a8c52c3e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:52:20.925466 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eafc191-a7af-4458-b1fe-5110a8c52c3e None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:52:20.943168 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 328/660] 10.222.0.40 () {56 vars in 1148 bytes} [Fri Aug 26 10:52:20 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 10:52:20.943168 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eafc191-a7af-4458-b1fe-5110a8c52c3e None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:52:20.992612 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae9e96b-a985-456f-b216-6595d1cfc4b5 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:52:21.015698 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:52:21.027037 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e29248-578b-45f8-9549-e2fbd6f1c884 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:52:21.027659 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e29248-578b-45f8-9549-e2fbd6f1c884 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:21.027933 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e29248-578b-45f8-9549-e2fbd6f1c884 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:21.028199 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e29248-578b-45f8-9549-e2fbd6f1c884 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:21.035127 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e29248-578b-45f8-9549-e2fbd6f1c884 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:52:21.036546 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e29248-578b-45f8-9549-e2fbd6f1c884 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:52:21.044150 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eafc191-a7af-4458-b1fe-5110a8c52c3e None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:52:21.047412 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application [None req-54e29248-578b-45f8-9549-e2fbd6f1c884 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 10:52:21.047412 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 10:52:21.047412 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 10:52:21.047412 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 10:52:21.047412 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 10:52:21.047412 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 10:52:21.047412 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 10:52:21.047412 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 10:52:21.047412 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 10:52:21.047412 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 10:52:21.047412 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 10:52:21.047412 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 10:52:21.047412 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 26 10:52:21.047412 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 26 10:52:21.047412 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 26 10:52:21.049007 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 26 10:52:21.049007 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 10:52:21.049007 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 10:52:21.049007 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 10:52:21.049007 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 10:52:21.049007 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 10:52:21.049007 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 10:52:21.049007 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 10:52:21.049007 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application with Lock( Aug 26 10:52:21.049007 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 10:52:21.049007 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._enter() Aug 26 10:52:21.049007 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 10:52:21.049007 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 10:52:21.049007 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 10:52:21.049007 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.creator() Aug 26 10:52:21.049007 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 10:52:21.049007 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application created_value = creator( Aug 26 10:52:21.049903 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 26 10:52:21.049903 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 26 10:52:21.049903 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 26 10:52:21.049903 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 26 10:52:21.049903 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 26 10:52:21.049903 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 26 10:52:21.049903 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 10:52:21.049903 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Aug 26 10:52:21.049903 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 333/661] 10.222.0.40 () {56 vars in 1111 bytes} [Fri Aug 26 10:52:20 2022] GET /identity/v3/projects/service => generated 88 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 10:52:21.050431 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e29248-578b-45f8-9549-e2fbd6f1c884 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:52:21.078519 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da020c65-8a22-416c-a9ab-489bb1b99a45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:52:21.079143 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da020c65-8a22-416c-a9ab-489bb1b99a45 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:21.079461 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da020c65-8a22-416c-a9ab-489bb1b99a45 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:21.079818 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da020c65-8a22-416c-a9ab-489bb1b99a45 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:21.133423 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:52:21.147515 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da020c65-8a22-416c-a9ab-489bb1b99a45 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:52:21.148912 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da020c65-8a22-416c-a9ab-489bb1b99a45 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:52:21.151558 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e29248-578b-45f8-9549-e2fbd6f1c884 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:52:21.185681 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581ef6bb-fa61-4b55-9923-cc4e0953a179 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:52:21.186332 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581ef6bb-fa61-4b55-9923-cc4e0953a179 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:21.186608 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581ef6bb-fa61-4b55-9923-cc4e0953a179 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:21.186873 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581ef6bb-fa61-4b55-9923-cc4e0953a179 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:21.194104 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-581ef6bb-fa61-4b55-9923-cc4e0953a179 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:52:21.195541 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-581ef6bb-fa61-4b55-9923-cc4e0953a179 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:52:21.206385 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application [None req-581ef6bb-fa61-4b55-9923-cc4e0953a179 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 26 10:52:21.206385 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 10:52:21.206385 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 10:52:21.206385 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 10:52:21.206385 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 10:52:21.206385 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 10:52:21.206385 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 10:52:21.206385 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 10:52:21.206385 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 10:52:21.206385 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 10:52:21.206385 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 10:52:21.206385 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 10:52:21.206385 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 26 10:52:21.206385 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 26 10:52:21.206385 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 26 10:52:21.206385 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 26 10:52:21.207328 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 10:52:21.207328 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 10:52:21.207328 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 26 10:52:21.207328 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 10:52:21.207328 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 26 10:52:21.207328 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 10:52:21.207328 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 10:52:21.207328 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 10:52:21.207328 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 10:52:21.207328 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 10:52:21.207328 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 10:52:21.207328 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application with Lock( Aug 26 10:52:21.207328 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 10:52:21.207328 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._enter() Aug 26 10:52:21.207328 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 10:52:21.207328 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 10:52:21.207328 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 10:52:21.207328 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.creator() Aug 26 10:52:21.208367 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 10:52:21.208367 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application created_value = creator( Aug 26 10:52:21.208367 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 26 10:52:21.208367 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 26 10:52:21.208367 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 26 10:52:21.208367 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 26 10:52:21.208367 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 26 10:52:21.208367 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 26 10:52:21.208367 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 26 10:52:21.208367 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Aug 26 10:52:21.208367 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 334/662] 10.222.0.40 () {56 vars in 1102 bytes} [Fri Aug 26 10:52:21 2022] GET /identity/v3/users/neutron => generated 85 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 10:52:21.208367 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581ef6bb-fa61-4b55-9923-cc4e0953a179 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:52:21.252077 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 329/663] 10.222.0.40 () {58 vars in 1282 bytes} [Fri Aug 26 10:52:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 26 10:52:21.252943 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da020c65-8a22-416c-a9ab-489bb1b99a45 None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:52:21.253250 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:52:21.310558 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581ef6bb-fa61-4b55-9923-cc4e0953a179 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:52:21.339732 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:52:21.342974 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b965c2d-1931-447c-bc98-20b3f3e77240 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:52:21.343580 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b965c2d-1931-447c-bc98-20b3f3e77240 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:21.343851 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b965c2d-1931-447c-bc98-20b3f3e77240 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:21.344114 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b965c2d-1931-447c-bc98-20b3f3e77240 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:21.345403 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b965c2d-1931-447c-bc98-20b3f3e77240 None admin] RBAC: Adding query filter params (role.id=3bda770fcd5c46cdbfabcd7404a25e66, scope.project.id=0e30ccf0c1204727b786876c141dbcd2, user.id=1f4b4a662db049ecacc865ecff10bdb0) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 10:52:21.345739 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b965c2d-1931-447c-bc98-20b3f3e77240 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:52:21.347330 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b965c2d-1931-447c-bc98-20b3f3e77240 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:52:21.352950 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 335/664] 10.222.0.40 () {56 vars in 1374 bytes} [Fri Aug 26 10:52:21 2022] GET /identity/v3/role_assignments?user.id=1f4b4a662db049ecacc865ecff10bdb0&scope.project.id=0e30ccf0c1204727b786876c141dbcd2&role.id=3bda770fcd5c46cdbfabcd7404a25e66 => generated 608 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 10:52:21.353720 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b965c2d-1931-447c-bc98-20b3f3e77240 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:52:21.364145 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da020c65-8a22-416c-a9ab-489bb1b99a45 None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:52:21.403789 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c0ad7a-f5bc-46b0-927e-0076e66866b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:52:21.404357 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c0ad7a-f5bc-46b0-927e-0076e66866b5 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:21.404633 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c0ad7a-f5bc-46b0-927e-0076e66866b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:21.404903 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c0ad7a-f5bc-46b0-927e-0076e66866b5 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:21.418014 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c0ad7a-f5bc-46b0-927e-0076e66866b5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 10:52:21.423775 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application [None req-16c0ad7a-f5bc-46b0-927e-0076e66866b5 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 26 10:52:21.423775 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 10:52:21.423775 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 10:52:21.423775 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 10:52:21.423775 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 10:52:21.423775 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 10:52:21.423775 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 10:52:21.423775 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 10:52:21.423775 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 10:52:21.423775 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 10:52:21.423775 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 10:52:21.423775 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 10:52:21.423775 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 10:52:21.423775 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 10:52:21.423775 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 10:52:21.423775 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 10:52:21.424757 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 10:52:21.424757 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 10:52:21.424757 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 10:52:21.424757 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 10:52:21.424757 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 10:52:21.424757 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 10:52:21.424757 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 10:52:21.424757 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 10:52:21.424757 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 10:52:21.424757 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 10:52:21.424757 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 10:52:21.424757 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application with Lock( Aug 26 10:52:21.424757 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 10:52:21.424757 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._enter() Aug 26 10:52:21.424757 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 10:52:21.424757 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 10:52:21.425628 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 10:52:21.425628 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.creator() Aug 26 10:52:21.425628 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 10:52:21.425628 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application created_value = creator( Aug 26 10:52:21.425628 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 10:52:21.425628 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 10:52:21.425628 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 10:52:21.425628 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 10:52:21.425628 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 10:52:21.425628 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 10:52:21.425628 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 26 10:52:21.425628 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Aug 26 10:52:21.430431 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 330/665] 10.222.0.40 () {56 vars in 1102 bytes} [Fri Aug 26 10:52:21 2022] GET /identity/v3/roles/service => generated 85 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 10:52:21.431401 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c0ad7a-f5bc-46b0-927e-0076e66866b5 None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:52:21.484494 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b965c2d-1931-447c-bc98-20b3f3e77240 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:52:21.496541 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:52:21.528507 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d61cdbd-7d75-408c-ad6a-07c1aa19816e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:52:21.529249 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d61cdbd-7d75-408c-ad6a-07c1aa19816e None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:21.529527 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d61cdbd-7d75-408c-ad6a-07c1aa19816e None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:21.529797 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d61cdbd-7d75-408c-ad6a-07c1aa19816e None admin] PATH_INFO: `/v3/projects` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:21.530564 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d61cdbd-7d75-408c-ad6a-07c1aa19816e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 10:52:21.530870 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d61cdbd-7d75-408c-ad6a-07c1aa19816e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:52:21.532884 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d61cdbd-7d75-408c-ad6a-07c1aa19816e None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:52:21.546552 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 336/666] 10.222.0.40 () {56 vars in 1148 bytes} [Fri Aug 26 10:52:21 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 10:52:21.547386 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d61cdbd-7d75-408c-ad6a-07c1aa19816e None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:52:21.549041 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c0ad7a-f5bc-46b0-927e-0076e66866b5 None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:52:21.592599 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33268cc6-72c8-4fac-8a54-17a4c17fa6e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:52:21.593213 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33268cc6-72c8-4fac-8a54-17a4c17fa6e3 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:21.593487 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33268cc6-72c8-4fac-8a54-17a4c17fa6e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:21.593756 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33268cc6-72c8-4fac-8a54-17a4c17fa6e3 None admin] PATH_INFO: `/v3/users` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:21.594529 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33268cc6-72c8-4fac-8a54-17a4c17fa6e3 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 10:52:21.595487 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33268cc6-72c8-4fac-8a54-17a4c17fa6e3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:52:21.597069 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33268cc6-72c8-4fac-8a54-17a4c17fa6e3 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:52:21.615714 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 331/667] 10.222.0.40 () {56 vars in 1139 bytes} [Fri Aug 26 10:52:21 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 10:52:21.618304 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33268cc6-72c8-4fac-8a54-17a4c17fa6e3 None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:52:21.625177 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:52:21.659873 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d61cdbd-7d75-408c-ad6a-07c1aa19816e None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:52:21.695477 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5929450-a0be-429b-b8b2-1c3c87b26212 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:52:21.696106 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5929450-a0be-429b-b8b2-1c3c87b26212 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:21.696380 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5929450-a0be-429b-b8b2-1c3c87b26212 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:21.696647 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5929450-a0be-429b-b8b2-1c3c87b26212 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:21.707623 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5929450-a0be-429b-b8b2-1c3c87b26212 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:52:21.709068 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5929450-a0be-429b-b8b2-1c3c87b26212 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:52:21.724476 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application [None req-d5929450-a0be-429b-b8b2-1c3c87b26212 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 10:52:21.724476 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 10:52:21.724476 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 10:52:21.724476 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 10:52:21.724476 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 10:52:21.724476 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 10:52:21.724476 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 10:52:21.724476 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 10:52:21.724476 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 10:52:21.724476 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 10:52:21.724476 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 10:52:21.724476 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 10:52:21.724476 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 10:52:21.724476 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 10:52:21.724476 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 10:52:21.724476 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 10:52:21.725492 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 10:52:21.725492 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 10:52:21.725492 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 10:52:21.725492 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 10:52:21.725492 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 10:52:21.725492 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 10:52:21.725492 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 10:52:21.725492 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application with Lock( Aug 26 10:52:21.725492 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 10:52:21.725492 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._enter() Aug 26 10:52:21.725492 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 10:52:21.725492 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 10:52:21.725492 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 10:52:21.725492 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.creator() Aug 26 10:52:21.725492 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 10:52:21.725492 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application created_value = creator( Aug 26 10:52:21.725492 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 10:52:21.726417 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 10:52:21.726417 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 10:52:21.726417 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Aug 26 10:52:21.726417 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 337/668] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Aug 26 10:52:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 10:52:21.726417 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5929450-a0be-429b-b8b2-1c3c87b26212 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:52:21.738290 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33268cc6-72c8-4fac-8a54-17a4c17fa6e3 None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:52:21.751210 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:52:21.778580 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29aa49c-793e-48da-adb4-375a16739ff5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:52:21.779186 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29aa49c-793e-48da-adb4-375a16739ff5 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:21.779467 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29aa49c-793e-48da-adb4-375a16739ff5 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:21.779736 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29aa49c-793e-48da-adb4-375a16739ff5 None admin] PATH_INFO: `/v3/roles` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:21.780395 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a29aa49c-793e-48da-adb4-375a16739ff5 None admin] RBAC: Adding query filter params (name=service) {{(pid=81423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 10:52:21.780688 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a29aa49c-793e-48da-adb4-375a16739ff5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:52:21.782180 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a29aa49c-793e-48da-adb4-375a16739ff5 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:52:21.796346 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 332/669] 10.222.0.40 () {56 vars in 1103 bytes} [Fri Aug 26 10:52:21 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 10:52:21.796346 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29aa49c-793e-48da-adb4-375a16739ff5 None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:52:21.848230 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5929450-a0be-429b-b8b2-1c3c87b26212 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:52:21.868619 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:52:21.891215 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13407305-819f-4a17-9f72-44297555ce76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:52:21.892533 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13407305-819f-4a17-9f72-44297555ce76 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:21.892994 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13407305-819f-4a17-9f72-44297555ce76 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:21.893367 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13407305-819f-4a17-9f72-44297555ce76 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:21.894122 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13407305-819f-4a17-9f72-44297555ce76 None admin] RBAC: Adding query filter params (role.id=3bda770fcd5c46cdbfabcd7404a25e66, scope.project.id=0e30ccf0c1204727b786876c141dbcd2, user.id=8631d7ecfd2849daa8694aecccd22a05) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 10:52:21.894430 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13407305-819f-4a17-9f72-44297555ce76 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:52:21.895853 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13407305-819f-4a17-9f72-44297555ce76 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:52:21.914397 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 338/670] 10.222.0.40 () {56 vars in 1374 bytes} [Fri Aug 26 10:52:21 2022] GET /identity/v3/role_assignments?user.id=8631d7ecfd2849daa8694aecccd22a05&scope.project.id=0e30ccf0c1204727b786876c141dbcd2&role.id=3bda770fcd5c46cdbfabcd7404a25e66 => generated 608 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 10:52:21.914397 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13407305-819f-4a17-9f72-44297555ce76 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:52:21.927542 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29aa49c-793e-48da-adb4-375a16739ff5 None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:52:21.969512 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b554c2-46ca-4749-aa23-0cad09e9b20f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:52:21.970735 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b554c2-46ca-4749-aa23-0cad09e9b20f None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:21.971119 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b554c2-46ca-4749-aa23-0cad09e9b20f None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:21.971394 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b554c2-46ca-4749-aa23-0cad09e9b20f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:22.001252 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:52:22.019538 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13407305-819f-4a17-9f72-44297555ce76 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:52:22.059658 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b554c2-46ca-4749-aa23-0cad09e9b20f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:52:22.062048 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b554c2-46ca-4749-aa23-0cad09e9b20f None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:52:22.072649 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d0cfd1-72bd-4ab6-bf81-c605cc449aa7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:52:22.073271 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d0cfd1-72bd-4ab6-bf81-c605cc449aa7 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:22.073551 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d0cfd1-72bd-4ab6-bf81-c605cc449aa7 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:22.074189 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d0cfd1-72bd-4ab6-bf81-c605cc449aa7 None admin] PATH_INFO: `/v3/domains` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:22.074534 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d0cfd1-72bd-4ab6-bf81-c605cc449aa7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 10:52:22.074837 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d0cfd1-72bd-4ab6-bf81-c605cc449aa7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:52:22.076217 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d0cfd1-72bd-4ab6-bf81-c605cc449aa7 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:52:22.089494 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 339/671] 10.222.0.40 () {56 vars in 1109 bytes} [Fri Aug 26 10:52:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 10:52:22.091250 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d0cfd1-72bd-4ab6-bf81-c605cc449aa7 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:52:22.143820 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:52:22.190941 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 333/672] 10.222.0.40 () {58 vars in 1282 bytes} [Fri Aug 26 10:52:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 398 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 26 10:52:22.191728 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b554c2-46ca-4749-aa23-0cad09e9b20f None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:52:22.211186 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d0cfd1-72bd-4ab6-bf81-c605cc449aa7 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:52:22.254976 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e77f3e3-85db-49f0-b449-6c58b029034b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:52:22.256062 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e77f3e3-85db-49f0-b449-6c58b029034b None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:22.256440 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e77f3e3-85db-49f0-b449-6c58b029034b None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:22.256974 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e77f3e3-85db-49f0-b449-6c58b029034b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:22.271784 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e77f3e3-85db-49f0-b449-6c58b029034b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:52:22.273465 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e77f3e3-85db-49f0-b449-6c58b029034b None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:52:22.279163 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:52:22.294433 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application [None req-7e77f3e3-85db-49f0-b449-6c58b029034b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 10:52:22.294433 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 10:52:22.294433 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 10:52:22.294433 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 10:52:22.294433 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 10:52:22.294433 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 10:52:22.294433 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 10:52:22.294433 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 10:52:22.294433 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 10:52:22.294433 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 10:52:22.294433 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 10:52:22.294433 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 10:52:22.294433 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 10:52:22.294433 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 10:52:22.294433 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 10:52:22.294433 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 10:52:22.295555 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 10:52:22.295555 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 10:52:22.295555 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 10:52:22.295555 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 10:52:22.295555 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 10:52:22.295555 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 10:52:22.295555 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 10:52:22.295555 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application with Lock( Aug 26 10:52:22.295555 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 10:52:22.295555 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._enter() Aug 26 10:52:22.295555 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 10:52:22.295555 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 10:52:22.295555 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 10:52:22.295555 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.creator() Aug 26 10:52:22.295555 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 10:52:22.295555 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application created_value = creator( Aug 26 10:52:22.295555 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 10:52:22.296466 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 10:52:22.296466 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 10:52:22.296466 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Aug 26 10:52:22.300278 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 340/673] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Aug 26 10:52:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 10:52:22.301234 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e77f3e3-85db-49f0-b449-6c58b029034b None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:52:22.310090 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b554c2-46ca-4749-aa23-0cad09e9b20f None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:52:22.361206 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3136e29b-880d-4309-801f-686af8929eeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:52:22.362897 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3136e29b-880d-4309-801f-686af8929eeb None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:22.363412 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3136e29b-880d-4309-801f-686af8929eeb None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:22.364186 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3136e29b-880d-4309-801f-686af8929eeb None admin] PATH_INFO: `/v3/projects/service` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:22.374447 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3136e29b-880d-4309-801f-686af8929eeb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:52:22.376731 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3136e29b-880d-4309-801f-686af8929eeb None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:52:22.395149 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application [None req-3136e29b-880d-4309-801f-686af8929eeb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 10:52:22.395149 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 10:52:22.395149 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 10:52:22.395149 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 10:52:22.395149 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 10:52:22.395149 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 10:52:22.395149 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 10:52:22.395149 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 10:52:22.395149 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 10:52:22.395149 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 10:52:22.395149 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 10:52:22.395149 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 10:52:22.395149 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 26 10:52:22.395149 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 26 10:52:22.395149 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 26 10:52:22.396092 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 26 10:52:22.396092 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 10:52:22.396092 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 10:52:22.396092 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 10:52:22.396092 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 10:52:22.396092 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 10:52:22.396092 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 10:52:22.396092 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 10:52:22.396092 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application with Lock( Aug 26 10:52:22.396092 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 10:52:22.396092 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._enter() Aug 26 10:52:22.396092 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 10:52:22.396092 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 10:52:22.396092 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 10:52:22.396092 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.creator() Aug 26 10:52:22.396092 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 10:52:22.396092 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application created_value = creator( Aug 26 10:52:22.396990 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 26 10:52:22.396990 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 26 10:52:22.396990 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 26 10:52:22.396990 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 26 10:52:22.396990 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 26 10:52:22.396990 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 26 10:52:22.396990 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 10:52:22.396990 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Aug 26 10:52:22.399377 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 334/674] 10.222.0.40 () {56 vars in 1111 bytes} [Fri Aug 26 10:52:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 10:52:22.400381 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3136e29b-880d-4309-801f-686af8929eeb None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:52:22.415540 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e77f3e3-85db-49f0-b449-6c58b029034b None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:52:22.432040 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:52:22.464021 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bcfa88d-cc8e-4267-9f81-1b2a2ba2a28e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:52:22.464567 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bcfa88d-cc8e-4267-9f81-1b2a2ba2a28e None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:22.464842 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bcfa88d-cc8e-4267-9f81-1b2a2ba2a28e None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:22.465109 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bcfa88d-cc8e-4267-9f81-1b2a2ba2a28e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:22.476542 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bcfa88d-cc8e-4267-9f81-1b2a2ba2a28e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:52:22.478010 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bcfa88d-cc8e-4267-9f81-1b2a2ba2a28e None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:52:22.493487 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application [None req-5bcfa88d-cc8e-4267-9f81-1b2a2ba2a28e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 10:52:22.493487 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 10:52:22.493487 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 10:52:22.493487 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 10:52:22.493487 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 10:52:22.493487 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 10:52:22.493487 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 10:52:22.493487 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 10:52:22.493487 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 10:52:22.493487 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 10:52:22.493487 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 10:52:22.493487 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 10:52:22.493487 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 10:52:22.493487 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 10:52:22.493487 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 10:52:22.493487 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 10:52:22.494499 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 10:52:22.494499 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 10:52:22.494499 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 10:52:22.494499 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 10:52:22.494499 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 10:52:22.494499 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 10:52:22.494499 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 10:52:22.494499 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application with Lock( Aug 26 10:52:22.494499 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 10:52:22.494499 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._enter() Aug 26 10:52:22.494499 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 10:52:22.494499 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 10:52:22.494499 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 10:52:22.494499 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.creator() Aug 26 10:52:22.494499 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 10:52:22.494499 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application created_value = creator( Aug 26 10:52:22.494499 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 10:52:22.495437 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 10:52:22.495437 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 10:52:22.495437 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Aug 26 10:52:22.495437 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 341/675] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Aug 26 10:52:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 10:52:22.496137 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bcfa88d-cc8e-4267-9f81-1b2a2ba2a28e None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:52:22.537184 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3136e29b-880d-4309-801f-686af8929eeb None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:52:22.549041 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:52:22.581867 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e225f93-cf2f-4b32-a269-5922d8bd8acf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:52:22.583191 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e225f93-cf2f-4b32-a269-5922d8bd8acf None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:22.584141 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e225f93-cf2f-4b32-a269-5922d8bd8acf None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:22.584759 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e225f93-cf2f-4b32-a269-5922d8bd8acf None admin] PATH_INFO: `/v3/domains` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:22.585739 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e225f93-cf2f-4b32-a269-5922d8bd8acf None admin] RBAC: Adding query filter params (name=Default) {{(pid=81423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 10:52:22.586534 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e225f93-cf2f-4b32-a269-5922d8bd8acf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:52:22.588538 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e225f93-cf2f-4b32-a269-5922d8bd8acf None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:52:22.602679 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bcfa88d-cc8e-4267-9f81-1b2a2ba2a28e None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:52:22.606470 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 335/676] 10.222.0.40 () {56 vars in 1109 bytes} [Fri Aug 26 10:52:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 10:52:22.607509 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e225f93-cf2f-4b32-a269-5922d8bd8acf None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:52:22.652261 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832dbf10-af27-45c7-be05-45c3abb38509 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:52:22.653255 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832dbf10-af27-45c7-be05-45c3abb38509 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:22.654365 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832dbf10-af27-45c7-be05-45c3abb38509 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:22.654844 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832dbf10-af27-45c7-be05-45c3abb38509 None admin] PATH_INFO: `/v3/projects` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:22.656211 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-832dbf10-af27-45c7-be05-45c3abb38509 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 10:52:22.656742 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-832dbf10-af27-45c7-be05-45c3abb38509 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:52:22.659439 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-832dbf10-af27-45c7-be05-45c3abb38509 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:52:22.680634 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 342/677] 10.222.0.40 () {56 vars in 1148 bytes} [Fri Aug 26 10:52:22 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 10:52:22.680634 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832dbf10-af27-45c7-be05-45c3abb38509 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:52:22.711172 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:52:22.728919 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e225f93-cf2f-4b32-a269-5922d8bd8acf None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:52:22.765165 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31be9a61-19c9-4221-bc89-544aa1eade53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:52:22.765904 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31be9a61-19c9-4221-bc89-544aa1eade53 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:22.766322 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31be9a61-19c9-4221-bc89-544aa1eade53 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:22.767403 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31be9a61-19c9-4221-bc89-544aa1eade53 None admin] PATH_INFO: `/v3/domains` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:22.768142 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31be9a61-19c9-4221-bc89-544aa1eade53 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 10:52:22.768593 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31be9a61-19c9-4221-bc89-544aa1eade53 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:52:22.770067 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31be9a61-19c9-4221-bc89-544aa1eade53 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:52:22.790106 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 336/678] 10.222.0.40 () {56 vars in 1109 bytes} [Fri Aug 26 10:52:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 10:52:22.791231 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31be9a61-19c9-4221-bc89-544aa1eade53 None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:52:22.811056 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832dbf10-af27-45c7-be05-45c3abb38509 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:52:22.823223 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:52:22.851539 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bafcf142-37a9-4dc2-9a6c-2ce6f7f96fc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:52:22.852325 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafcf142-37a9-4dc2-9a6c-2ce6f7f96fc5 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:22.852751 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafcf142-37a9-4dc2-9a6c-2ce6f7f96fc5 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:22.853053 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafcf142-37a9-4dc2-9a6c-2ce6f7f96fc5 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:22.861465 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bafcf142-37a9-4dc2-9a6c-2ce6f7f96fc5 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:52:22.863056 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bafcf142-37a9-4dc2-9a6c-2ce6f7f96fc5 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:52:22.879051 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application [None req-bafcf142-37a9-4dc2-9a6c-2ce6f7f96fc5 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 26 10:52:22.879051 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 10:52:22.879051 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 10:52:22.879051 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 10:52:22.879051 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 10:52:22.879051 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 10:52:22.879051 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 10:52:22.879051 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 10:52:22.879051 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 10:52:22.879051 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 10:52:22.879051 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 10:52:22.879051 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 10:52:22.879051 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 26 10:52:22.879051 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 26 10:52:22.879051 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 26 10:52:22.879051 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 26 10:52:22.880212 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 10:52:22.880212 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 10:52:22.880212 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 26 10:52:22.880212 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 10:52:22.880212 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 26 10:52:22.880212 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 10:52:22.880212 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 10:52:22.880212 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 10:52:22.880212 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 10:52:22.880212 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 10:52:22.880212 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 10:52:22.880212 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application with Lock( Aug 26 10:52:22.880212 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 10:52:22.880212 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._enter() Aug 26 10:52:22.880212 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 10:52:22.880212 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 10:52:22.880212 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 10:52:22.880212 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.creator() Aug 26 10:52:22.881359 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 10:52:22.881359 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application created_value = creator( Aug 26 10:52:22.881359 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 26 10:52:22.881359 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 26 10:52:22.881359 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 26 10:52:22.881359 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 26 10:52:22.881359 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 26 10:52:22.881359 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 26 10:52:22.881359 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 26 10:52:22.881359 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Aug 26 10:52:22.882814 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 343/679] 10.222.0.40 () {56 vars in 1096 bytes} [Fri Aug 26 10:52:22 2022] GET /identity/v3/users/swift => generated 83 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 10:52:22.884809 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bafcf142-37a9-4dc2-9a6c-2ce6f7f96fc5 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:52:22.933638 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31be9a61-19c9-4221-bc89-544aa1eade53 None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:52:22.979980 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04cf7a3-941a-4d05-8dc9-ff48dd4a8cf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:52:22.980719 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04cf7a3-941a-4d05-8dc9-ff48dd4a8cf2 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:22.981087 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04cf7a3-941a-4d05-8dc9-ff48dd4a8cf2 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:22.981490 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04cf7a3-941a-4d05-8dc9-ff48dd4a8cf2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:22.982673 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c04cf7a3-941a-4d05-8dc9-ff48dd4a8cf2 None admin] RBAC: Adding query filter params (role.id=3bda770fcd5c46cdbfabcd7404a25e66, scope.project.id=0e30ccf0c1204727b786876c141dbcd2, user.id=c277338a7f814cc6b01c5419775031bf) {{(pid=81423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 10:52:22.982795 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c04cf7a3-941a-4d05-8dc9-ff48dd4a8cf2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:52:22.984744 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c04cf7a3-941a-4d05-8dc9-ff48dd4a8cf2 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:52:22.998018 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 337/680] 10.222.0.40 () {56 vars in 1374 bytes} [Fri Aug 26 10:52:22 2022] GET /identity/v3/role_assignments?user.id=c277338a7f814cc6b01c5419775031bf&scope.project.id=0e30ccf0c1204727b786876c141dbcd2&role.id=3bda770fcd5c46cdbfabcd7404a25e66 => generated 608 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 10:52:22.998018 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04cf7a3-941a-4d05-8dc9-ff48dd4a8cf2 None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:52:23.010070 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bafcf142-37a9-4dc2-9a6c-2ce6f7f96fc5 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:52:23.025014 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:52:23.083781 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a69cd0d-1622-4053-a11d-aa1e8a4c3b0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:52:23.084722 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a69cd0d-1622-4053-a11d-aa1e8a4c3b0d None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:23.085106 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a69cd0d-1622-4053-a11d-aa1e8a4c3b0d None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:23.085470 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a69cd0d-1622-4053-a11d-aa1e8a4c3b0d None admin] PATH_INFO: `/v3/projects/service` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:23.099667 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a69cd0d-1622-4053-a11d-aa1e8a4c3b0d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:52:23.103790 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a69cd0d-1622-4053-a11d-aa1e8a4c3b0d None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:52:23.126404 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application [None req-4a69cd0d-1622-4053-a11d-aa1e8a4c3b0d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 10:52:23.126404 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 10:52:23.126404 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 10:52:23.126404 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 10:52:23.126404 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 10:52:23.126404 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 10:52:23.126404 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 10:52:23.126404 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 10:52:23.126404 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 10:52:23.126404 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 10:52:23.126404 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 10:52:23.126404 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 10:52:23.126404 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 26 10:52:23.126404 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 26 10:52:23.126404 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 26 10:52:23.127579 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 26 10:52:23.127579 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 10:52:23.127579 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 10:52:23.127579 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 10:52:23.127579 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 10:52:23.127579 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 10:52:23.127579 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 10:52:23.127579 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 10:52:23.127579 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application with Lock( Aug 26 10:52:23.127579 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 10:52:23.127579 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._enter() Aug 26 10:52:23.127579 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 10:52:23.127579 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 10:52:23.127579 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 10:52:23.127579 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.creator() Aug 26 10:52:23.127579 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 10:52:23.127579 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application created_value = creator( Aug 26 10:52:23.128675 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 26 10:52:23.128675 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 26 10:52:23.128675 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 26 10:52:23.128675 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 26 10:52:23.128675 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 26 10:52:23.128675 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 26 10:52:23.128675 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 10:52:23.128675 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Aug 26 10:52:23.131255 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 344/681] 10.222.0.40 () {56 vars in 1111 bytes} [Fri Aug 26 10:52:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 10:52:23.137361 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a69cd0d-1622-4053-a11d-aa1e8a4c3b0d None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:52:23.190085 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:52:23.218628 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04cf7a3-941a-4d05-8dc9-ff48dd4a8cf2 None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:52:23.263219 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56407d7d-b428-404c-8044-92be5f9327da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:52:23.263703 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56407d7d-b428-404c-8044-92be5f9327da None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:23.264073 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56407d7d-b428-404c-8044-92be5f9327da None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:23.264444 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56407d7d-b428-404c-8044-92be5f9327da None admin] PATH_INFO: `/v3/users` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:23.265258 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56407d7d-b428-404c-8044-92be5f9327da None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 10:52:23.265655 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56407d7d-b428-404c-8044-92be5f9327da None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:52:23.268816 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56407d7d-b428-404c-8044-92be5f9327da None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:52:23.290397 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a69cd0d-1622-4053-a11d-aa1e8a4c3b0d None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:52:23.292299 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 338/682] 10.222.0.40 () {56 vars in 1135 bytes} [Fri Aug 26 10:52:22 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 296 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 10:52:23.295519 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56407d7d-b428-404c-8044-92be5f9327da None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:52:23.339315 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9cba5e9-297d-4760-9bfd-78a1c4ea3092 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:52:23.340201 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9cba5e9-297d-4760-9bfd-78a1c4ea3092 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:23.340801 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9cba5e9-297d-4760-9bfd-78a1c4ea3092 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:23.341251 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9cba5e9-297d-4760-9bfd-78a1c4ea3092 None admin] PATH_INFO: `/v3/projects` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:23.342929 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9cba5e9-297d-4760-9bfd-78a1c4ea3092 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 10:52:23.343413 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9cba5e9-297d-4760-9bfd-78a1c4ea3092 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:52:23.345025 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9cba5e9-297d-4760-9bfd-78a1c4ea3092 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:52:23.359595 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 345/683] 10.222.0.40 () {56 vars in 1148 bytes} [Fri Aug 26 10:52:23 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 10:52:23.365659 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9cba5e9-297d-4760-9bfd-78a1c4ea3092 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:52:23.386567 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:52:23.431880 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56407d7d-b428-404c-8044-92be5f9327da None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:52:23.475543 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc84139-084f-4742-9e0d-992598897b96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:52:23.476406 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc84139-084f-4742-9e0d-992598897b96 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:23.476681 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc84139-084f-4742-9e0d-992598897b96 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:23.477324 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc84139-084f-4742-9e0d-992598897b96 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:23.480495 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:52:23.498216 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9cba5e9-297d-4760-9bfd-78a1c4ea3092 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:52:23.535685 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50fec0cb-dd53-433e-b586-0cadac75b2ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:52:23.536215 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fec0cb-dd53-433e-b586-0cadac75b2ce None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:23.536490 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fec0cb-dd53-433e-b586-0cadac75b2ce None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:23.536752 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fec0cb-dd53-433e-b586-0cadac75b2ce None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:23.537447 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50fec0cb-dd53-433e-b586-0cadac75b2ce None admin] RBAC: Adding query filter params (role.id=3bda770fcd5c46cdbfabcd7404a25e66, scope.project.id=0e30ccf0c1204727b786876c141dbcd2, user.id=91da2acd2534423091537e1ba068d2b7) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 10:52:23.537747 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50fec0cb-dd53-433e-b586-0cadac75b2ce None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:52:23.539262 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50fec0cb-dd53-433e-b586-0cadac75b2ce None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:52:23.546720 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 346/684] 10.222.0.40 () {56 vars in 1374 bytes} [Fri Aug 26 10:52:23 2022] GET /identity/v3/role_assignments?user.id=91da2acd2534423091537e1ba068d2b7&scope.project.id=0e30ccf0c1204727b786876c141dbcd2&role.id=3bda770fcd5c46cdbfabcd7404a25e66 => generated 608 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 10:52:23.548029 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39c64d4-d86a-4d87-af3e-34aeae88e6dd None None] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:23.548370 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39c64d4-d86a-4d87-af3e-34aeae88e6dd None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:23.548645 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39c64d4-d86a-4d87-af3e-34aeae88e6dd None None] PATH_INFO: `/` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:23.550144 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 347/685] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 26 10:52:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 10:52:23.555617 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d14a225-1638-4a34-9b05-5e7a3ccab46e None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:23.556027 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d14a225-1638-4a34-9b05-5e7a3ccab46e None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:23.556393 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d14a225-1638-4a34-9b05-5e7a3ccab46e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:23.572917 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc84139-084f-4742-9e0d-992598897b96 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:52:23.574414 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc84139-084f-4742-9e0d-992598897b96 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:52:23.576453 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-1d14a225-1638-4a34-9b05-5e7a3ccab46e None None] MFA Rules not processed for user `ca155a48e2024c32af3fd2e19e5fac11`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 10:52:23.577750 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-1d14a225-1638-4a34-9b05-5e7a3ccab46e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 10:52:23.720891 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 348/686] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 26 10:52:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 10:52:23.726191 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:52:23.757842 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 339/687] 10.222.0.40 () {58 vars in 1282 bytes} [Fri Aug 26 10:52:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 463 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 26 10:52:23.763571 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d14a225-1638-4a34-9b05-5e7a3ccab46e None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:52:23.884556 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:52:23.933452 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d14a225-1638-4a34-9b05-5e7a3ccab46e None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:52:24.016485 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e59c09f-d5a2-4628-afed-5baee7d0f771 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:52:24.018815 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e59c09f-d5a2-4628-afed-5baee7d0f771 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:24.019582 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e59c09f-d5a2-4628-afed-5baee7d0f771 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:24.020317 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e59c09f-d5a2-4628-afed-5baee7d0f771 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:24.027900 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc84139-084f-4742-9e0d-992598897b96 None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:52:24.044200 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e59c09f-d5a2-4628-afed-5baee7d0f771 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:52:24.047186 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e59c09f-d5a2-4628-afed-5baee7d0f771 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:52:24.074932 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application [None req-2e59c09f-d5a2-4628-afed-5baee7d0f771 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 10:52:24.074932 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 10:52:24.074932 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 10:52:24.074932 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 10:52:24.074932 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 10:52:24.074932 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 10:52:24.074932 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 10:52:24.074932 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 10:52:24.074932 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 10:52:24.074932 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 10:52:24.074932 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 10:52:24.074932 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 10:52:24.074932 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 10:52:24.074932 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 10:52:24.074932 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 10:52:24.074932 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 10:52:24.076761 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 10:52:24.076761 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 10:52:24.076761 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 10:52:24.076761 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 10:52:24.076761 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 10:52:24.076761 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 10:52:24.076761 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 10:52:24.076761 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application with Lock( Aug 26 10:52:24.076761 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 10:52:24.076761 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._enter() Aug 26 10:52:24.076761 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 10:52:24.076761 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 10:52:24.076761 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 10:52:24.076761 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.creator() Aug 26 10:52:24.076761 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 10:52:24.076761 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application created_value = creator( Aug 26 10:52:24.076761 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 10:52:24.078533 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 10:52:24.078533 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 10:52:24.078533 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Aug 26 10:52:24.086173 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 349/688] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Aug 26 10:52:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 319 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 10:52:24.090999 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e59c09f-d5a2-4628-afed-5baee7d0f771 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:52:24.205168 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc84139-084f-4742-9e0d-992598897b96 None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:52:24.254314 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e59c09f-d5a2-4628-afed-5baee7d0f771 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:52:24.261427 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8f4856-ceaf-44b3-94e3-36cfe14edc95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:52:24.263016 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8f4856-ceaf-44b3-94e3-36cfe14edc95 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:24.263808 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8f4856-ceaf-44b3-94e3-36cfe14edc95 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:24.264694 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8f4856-ceaf-44b3-94e3-36cfe14edc95 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:24.283966 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e8f4856-ceaf-44b3-94e3-36cfe14edc95 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 10:52:24.291954 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4110fba0-46fd-48f9-8901-41dbb2cda837 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:52:24.292564 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4110fba0-46fd-48f9-8901-41dbb2cda837 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:24.292843 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4110fba0-46fd-48f9-8901-41dbb2cda837 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:24.293152 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4110fba0-46fd-48f9-8901-41dbb2cda837 None admin] PATH_INFO: `/v3/domains` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:24.293754 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4110fba0-46fd-48f9-8901-41dbb2cda837 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 10:52:24.294085 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4110fba0-46fd-48f9-8901-41dbb2cda837 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:52:24.295467 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4110fba0-46fd-48f9-8901-41dbb2cda837 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:52:24.296066 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application [None req-1e8f4856-ceaf-44b3-94e3-36cfe14edc95 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 26 10:52:24.296066 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 10:52:24.296066 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 10:52:24.296066 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 10:52:24.296066 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 10:52:24.296066 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 10:52:24.296066 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 10:52:24.296066 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 10:52:24.296066 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 10:52:24.296066 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 10:52:24.296066 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 10:52:24.296066 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 10:52:24.296066 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 10:52:24.296066 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 10:52:24.296066 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 10:52:24.296066 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 10:52:24.297730 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 10:52:24.297730 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 10:52:24.297730 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 10:52:24.297730 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 10:52:24.297730 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 10:52:24.297730 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 10:52:24.297730 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 10:52:24.297730 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 10:52:24.297730 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 10:52:24.297730 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 10:52:24.297730 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 10:52:24.297730 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application with Lock( Aug 26 10:52:24.297730 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 10:52:24.297730 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._enter() Aug 26 10:52:24.297730 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 10:52:24.297730 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 10:52:24.299928 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 10:52:24.299928 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.creator() Aug 26 10:52:24.299928 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 10:52:24.299928 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application created_value = creator( Aug 26 10:52:24.299928 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 10:52:24.299928 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 10:52:24.299928 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 10:52:24.299928 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 10:52:24.299928 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 10:52:24.299928 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 10:52:24.299928 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 26 10:52:24.299928 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Aug 26 10:52:24.304051 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:52:24.307152 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 340/689] 10.222.0.40 () {56 vars in 1096 bytes} [Fri Aug 26 10:52:24 2022] GET /identity/v3/roles/admin => generated 83 bytes in 279 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 10:52:24.309465 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 350/690] 10.222.0.40 () {56 vars in 1109 bytes} [Fri Aug 26 10:52:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 10:52:24.315986 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4110fba0-46fd-48f9-8901-41dbb2cda837 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:52:24.316462 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90113fff-9f05-4390-a159-8277b18550bd None None] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:24.316777 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90113fff-9f05-4390-a159-8277b18550bd None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:24.317078 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90113fff-9f05-4390-a159-8277b18550bd None None] PATH_INFO: `/` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:24.318996 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 341/691] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 26 10:52:24 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 10:52:24.319740 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90113fff-9f05-4390-a159-8277b18550bd None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:52:24.359040 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:52:24.447032 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90113fff-9f05-4390-a159-8277b18550bd None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:52:24.494707 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab297be-388c-4659-a96d-6c0eaaf08dde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:52:24.495246 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab297be-388c-4659-a96d-6c0eaaf08dde None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:24.495525 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab297be-388c-4659-a96d-6c0eaaf08dde None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:24.495800 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab297be-388c-4659-a96d-6c0eaaf08dde None admin] PATH_INFO: `/v3/projects/service` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:24.498884 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4110fba0-46fd-48f9-8901-41dbb2cda837 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:52:24.504082 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab297be-388c-4659-a96d-6c0eaaf08dde None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:52:24.505538 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab297be-388c-4659-a96d-6c0eaaf08dde None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:52:24.518914 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application [None req-2ab297be-388c-4659-a96d-6c0eaaf08dde None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 10:52:24.518914 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 10:52:24.518914 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 10:52:24.518914 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 10:52:24.518914 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 10:52:24.518914 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 10:52:24.518914 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 10:52:24.518914 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 10:52:24.518914 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 10:52:24.518914 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 10:52:24.518914 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 10:52:24.518914 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 10:52:24.518914 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 26 10:52:24.518914 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 26 10:52:24.518914 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 26 10:52:24.520686 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 26 10:52:24.520686 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 10:52:24.520686 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 10:52:24.520686 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 10:52:24.520686 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 10:52:24.520686 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 10:52:24.520686 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 10:52:24.520686 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 10:52:24.520686 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application with Lock( Aug 26 10:52:24.520686 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 10:52:24.520686 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._enter() Aug 26 10:52:24.520686 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 10:52:24.520686 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 10:52:24.520686 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 10:52:24.520686 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.creator() Aug 26 10:52:24.520686 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 10:52:24.520686 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application created_value = creator( Aug 26 10:52:24.522480 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 26 10:52:24.522480 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 26 10:52:24.522480 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 26 10:52:24.522480 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 26 10:52:24.522480 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 26 10:52:24.522480 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 26 10:52:24.522480 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 10:52:24.522480 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Aug 26 10:52:24.522480 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 342/692] 10.222.0.40 () {56 vars in 1111 bytes} [Fri Aug 26 10:52:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 10:52:24.522480 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17cce94-023f-4dbb-81e6-1ea292470d6f None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:24.522480 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17cce94-023f-4dbb-81e6-1ea292470d6f None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:24.523621 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17cce94-023f-4dbb-81e6-1ea292470d6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:24.541867 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-c17cce94-023f-4dbb-81e6-1ea292470d6f None None] MFA Rules not processed for user `ca155a48e2024c32af3fd2e19e5fac11`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 10:52:24.543407 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-c17cce94-023f-4dbb-81e6-1ea292470d6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 10:52:24.576719 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4305a01-b517-466e-8784-ccfcff926fc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:52:24.578209 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4305a01-b517-466e-8784-ccfcff926fc1 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:24.578982 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4305a01-b517-466e-8784-ccfcff926fc1 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:24.580369 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4305a01-b517-466e-8784-ccfcff926fc1 None admin] PATH_INFO: `/v3/roles` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:24.582048 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4305a01-b517-466e-8784-ccfcff926fc1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 10:52:24.582999 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4305a01-b517-466e-8784-ccfcff926fc1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:52:24.586039 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4305a01-b517-466e-8784-ccfcff926fc1 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:52:24.602783 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 351/693] 10.222.0.40 () {56 vars in 1099 bytes} [Fri Aug 26 10:52:24 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 10:52:24.604717 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4305a01-b517-466e-8784-ccfcff926fc1 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:52:24.617600 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 343/694] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 26 10:52:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 10:52:24.618465 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17cce94-023f-4dbb-81e6-1ea292470d6f None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:52:24.658590 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:52:24.716779 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:52:24.794064 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4305a01-b517-466e-8784-ccfcff926fc1 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:52:24.806199 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17cce94-023f-4dbb-81e6-1ea292470d6f None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:52:24.875989 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281e5f2c-ccd5-47b3-bf04-78105882d100 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:52:24.876906 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281e5f2c-ccd5-47b3-bf04-78105882d100 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:24.877391 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281e5f2c-ccd5-47b3-bf04-78105882d100 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:24.877864 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281e5f2c-ccd5-47b3-bf04-78105882d100 None admin] PATH_INFO: `/v3/projects` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:24.878852 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-281e5f2c-ccd5-47b3-bf04-78105882d100 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 10:52:24.879364 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-281e5f2c-ccd5-47b3-bf04-78105882d100 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:52:24.881098 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-281e5f2c-ccd5-47b3-bf04-78105882d100 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:52:24.892108 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e2f0d9-4563-43c4-9eaf-c4962aa45aba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:52:24.893067 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e2f0d9-4563-43c4-9eaf-c4962aa45aba None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:24.893553 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e2f0d9-4563-43c4-9eaf-c4962aa45aba None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:24.894054 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e2f0d9-4563-43c4-9eaf-c4962aa45aba None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:24.905327 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 352/695] 10.222.0.40 () {56 vars in 1148 bytes} [Fri Aug 26 10:52:24 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 302 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 10:52:24.909794 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e2f0d9-4563-43c4-9eaf-c4962aa45aba None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:52:24.911621 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e2f0d9-4563-43c4-9eaf-c4962aa45aba None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:52:24.914201 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281e5f2c-ccd5-47b3-bf04-78105882d100 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:52:24.944369 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application [None req-44e2f0d9-4563-43c4-9eaf-c4962aa45aba None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 10:52:24.944369 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 10:52:24.944369 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 10:52:24.944369 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 10:52:24.944369 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 10:52:24.944369 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 10:52:24.944369 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 10:52:24.944369 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 10:52:24.944369 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 10:52:24.944369 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 10:52:24.944369 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 10:52:24.944369 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 10:52:24.944369 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 10:52:24.944369 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 10:52:24.944369 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 10:52:24.944369 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 10:52:24.945445 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 10:52:24.945445 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 10:52:24.945445 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 10:52:24.945445 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 10:52:24.945445 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 10:52:24.945445 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 10:52:24.945445 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 10:52:24.945445 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application with Lock( Aug 26 10:52:24.945445 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 10:52:24.945445 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._enter() Aug 26 10:52:24.945445 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 10:52:24.945445 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 10:52:24.945445 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 10:52:24.945445 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.creator() Aug 26 10:52:24.945445 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 10:52:24.945445 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application created_value = creator( Aug 26 10:52:24.945445 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 10:52:24.946394 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 10:52:24.946394 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 10:52:24.946394 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Aug 26 10:52:24.949660 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 344/696] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Aug 26 10:52:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 331 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 10:52:24.950885 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e2f0d9-4563-43c4-9eaf-c4962aa45aba None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:52:25.029672 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281e5f2c-ccd5-47b3-bf04-78105882d100 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:52:25.030313 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:52:25.047131 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:52:25.059925 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e2f0d9-4563-43c4-9eaf-c4962aa45aba None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:52:25.066353 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af437f9-3f53-4734-9189-6d83c52a2e7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:52:25.066965 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af437f9-3f53-4734-9189-6d83c52a2e7b None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:25.067242 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af437f9-3f53-4734-9189-6d83c52a2e7b None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:25.067512 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af437f9-3f53-4734-9189-6d83c52a2e7b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:25.068240 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af437f9-3f53-4734-9189-6d83c52a2e7b None admin] RBAC: Adding query filter params (role.id=3bda770fcd5c46cdbfabcd7404a25e66, scope.project.id=0e30ccf0c1204727b786876c141dbcd2, user.id=6529bea296c9482a973ced5fe9f2212f) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 10:52:25.068547 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af437f9-3f53-4734-9189-6d83c52a2e7b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:52:25.070012 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af437f9-3f53-4734-9189-6d83c52a2e7b None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:52:25.082394 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 353/697] 10.222.0.40 () {56 vars in 1374 bytes} [Fri Aug 26 10:52:24 2022] GET /identity/v3/role_assignments?user.id=6529bea296c9482a973ced5fe9f2212f&scope.project.id=0e30ccf0c1204727b786876c141dbcd2&role.id=3bda770fcd5c46cdbfabcd7404a25e66 => generated 608 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 10:52:25.083598 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af437f9-3f53-4734-9189-6d83c52a2e7b None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:52:25.100347 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66010f0-9847-4836-ba38-4c58536a82a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:52:25.106007 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66010f0-9847-4836-ba38-4c58536a82a7 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:25.106007 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66010f0-9847-4836-ba38-4c58536a82a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:25.106007 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66010f0-9847-4836-ba38-4c58536a82a7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:25.115522 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e66010f0-9847-4836-ba38-4c58536a82a7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:52:25.117151 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e66010f0-9847-4836-ba38-4c58536a82a7 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:52:25.134407 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application [None req-e66010f0-9847-4836-ba38-4c58536a82a7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 10:52:25.134407 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 10:52:25.134407 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 10:52:25.134407 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 10:52:25.134407 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 10:52:25.134407 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 10:52:25.134407 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 10:52:25.134407 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 10:52:25.134407 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 10:52:25.134407 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 10:52:25.134407 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 10:52:25.134407 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 10:52:25.134407 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 10:52:25.134407 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 10:52:25.134407 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 10:52:25.134407 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 10:52:25.135568 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 10:52:25.135568 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 10:52:25.135568 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 10:52:25.135568 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 10:52:25.135568 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 10:52:25.135568 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 10:52:25.135568 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 10:52:25.135568 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application with Lock( Aug 26 10:52:25.135568 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 10:52:25.135568 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._enter() Aug 26 10:52:25.135568 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 10:52:25.135568 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 10:52:25.135568 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 10:52:25.135568 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.creator() Aug 26 10:52:25.135568 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 10:52:25.135568 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application created_value = creator( Aug 26 10:52:25.135568 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 10:52:25.136643 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 10:52:25.136643 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 10:52:25.136643 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Aug 26 10:52:25.138154 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 345/698] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Aug 26 10:52:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 10:52:25.147078 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66010f0-9847-4836-ba38-4c58536a82a7 None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:52:25.294395 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af437f9-3f53-4734-9189-6d83c52a2e7b None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:52:25.332141 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66010f0-9847-4836-ba38-4c58536a82a7 None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:52:25.336793 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d524a382-f806-4e6d-8c8f-9fd4ab1f18d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:52:25.337696 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d524a382-f806-4e6d-8c8f-9fd4ab1f18d9 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:25.338243 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d524a382-f806-4e6d-8c8f-9fd4ab1f18d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:25.338728 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d524a382-f806-4e6d-8c8f-9fd4ab1f18d9 None admin] PATH_INFO: `/v3/domains` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:25.339581 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d524a382-f806-4e6d-8c8f-9fd4ab1f18d9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 10:52:25.340102 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d524a382-f806-4e6d-8c8f-9fd4ab1f18d9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:52:25.343156 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d524a382-f806-4e6d-8c8f-9fd4ab1f18d9 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:52:25.362796 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 354/699] 10.222.0.40 () {56 vars in 1109 bytes} [Fri Aug 26 10:52:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 10:52:25.365447 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:52:25.368468 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d524a382-f806-4e6d-8c8f-9fd4ab1f18d9 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:52:25.384282 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54a3c9d-b846-4b8b-af9c-6362a619c825 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:52:25.385221 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54a3c9d-b846-4b8b-af9c-6362a619c825 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:25.385507 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54a3c9d-b846-4b8b-af9c-6362a619c825 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:25.385785 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54a3c9d-b846-4b8b-af9c-6362a619c825 None admin] PATH_INFO: `/v3/domains` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:25.386793 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a54a3c9d-b846-4b8b-af9c-6362a619c825 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 10:52:25.387103 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a54a3c9d-b846-4b8b-af9c-6362a619c825 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:52:25.388492 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a54a3c9d-b846-4b8b-af9c-6362a619c825 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:52:25.407323 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 346/700] 10.222.0.40 () {56 vars in 1109 bytes} [Fri Aug 26 10:52:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 10:52:25.413377 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54a3c9d-b846-4b8b-af9c-6362a619c825 None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:52:25.429415 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:52:25.510840 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d524a382-f806-4e6d-8c8f-9fd4ab1f18d9 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:52:25.529910 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54a3c9d-b846-4b8b-af9c-6362a619c825 None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:52:25.556596 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13507346-b2ff-406b-a97e-0f1ef98d3ed4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:52:25.557747 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13507346-b2ff-406b-a97e-0f1ef98d3ed4 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:25.558591 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13507346-b2ff-406b-a97e-0f1ef98d3ed4 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:25.559084 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13507346-b2ff-406b-a97e-0f1ef98d3ed4 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:25.572562 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0e23f3-6274-4e72-a3ab-31fb7183ea9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:52:25.573241 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0e23f3-6274-4e72-a3ab-31fb7183ea9c None admin] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:25.573524 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0e23f3-6274-4e72-a3ab-31fb7183ea9c None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:25.573794 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0e23f3-6274-4e72-a3ab-31fb7183ea9c None admin] PATH_INFO: `/v3/users` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:25.574512 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe0e23f3-6274-4e72-a3ab-31fb7183ea9c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:52:25.575972 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe0e23f3-6274-4e72-a3ab-31fb7183ea9c None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:52:25.605018 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 347/701] 10.222.0.40 () {60 vars in 1132 bytes} [Fri Aug 26 10:52:25 2022] POST /identity/v3/users => generated 254 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 10:52:25.694694 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13507346-b2ff-406b-a97e-0f1ef98d3ed4 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:52:25.696733 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13507346-b2ff-406b-a97e-0f1ef98d3ed4 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:52:25.705878 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e14e92c-600a-4afb-a7bb-45793f644aa5 None None] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:25.706257 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e14e92c-600a-4afb-a7bb-45793f644aa5 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:25.706571 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e14e92c-600a-4afb-a7bb-45793f644aa5 None None] PATH_INFO: `/` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:25.708269 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 348/702] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 26 10:52:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 26 10:52:25.712005 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application [None req-13507346-b2ff-406b-a97e-0f1ef98d3ed4 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 26 10:52:25.712005 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 10:52:25.712005 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 10:52:25.712005 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 10:52:25.712005 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 10:52:25.712005 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 10:52:25.712005 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 10:52:25.712005 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 10:52:25.712005 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 10:52:25.712005 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 10:52:25.712005 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 10:52:25.712005 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 10:52:25.712005 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 26 10:52:25.712005 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 26 10:52:25.712005 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 26 10:52:25.712005 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 26 10:52:25.713141 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 10:52:25.713141 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 10:52:25.713141 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 26 10:52:25.713141 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 10:52:25.713141 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 26 10:52:25.713141 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 10:52:25.713141 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 10:52:25.713141 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 10:52:25.713141 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 10:52:25.713141 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 10:52:25.713141 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 10:52:25.713141 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application with Lock( Aug 26 10:52:25.713141 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 10:52:25.713141 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._enter() Aug 26 10:52:25.713141 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 10:52:25.713141 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 10:52:25.713141 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 10:52:25.713141 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.creator() Aug 26 10:52:25.714360 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 10:52:25.714360 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application created_value = creator( Aug 26 10:52:25.714360 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 26 10:52:25.714360 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 26 10:52:25.714360 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 26 10:52:25.714360 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 26 10:52:25.714360 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 26 10:52:25.714360 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 26 10:52:25.714360 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 26 10:52:25.714360 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Aug 26 10:52:25.716574 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 355/703] 10.222.0.40 () {56 vars in 1093 bytes} [Fri Aug 26 10:52:25 2022] GET /identity/v3/users/nova => generated 82 bytes in 348 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 10:52:25.720590 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba9324e-3805-48da-8d93-9108b9da6c62 None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:25.720874 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba9324e-3805-48da-8d93-9108b9da6c62 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:25.721186 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba9324e-3805-48da-8d93-9108b9da6c62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:25.722785 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13507346-b2ff-406b-a97e-0f1ef98d3ed4 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:52:25.763190 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-7ba9324e-3805-48da-8d93-9108b9da6c62 None None] MFA Rules not processed for user `ca155a48e2024c32af3fd2e19e5fac11`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 10:52:25.765126 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-7ba9324e-3805-48da-8d93-9108b9da6c62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 10:52:25.822964 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:52:25.835118 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:52:25.839189 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13507346-b2ff-406b-a97e-0f1ef98d3ed4 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:52:25.872020 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 349/704] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 26 10:52:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 10:52:25.889787 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5e9648-8656-4cb6-ab2d-e1ce99e7a7e8 None None] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:25.890129 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5e9648-8656-4cb6-ab2d-e1ce99e7a7e8 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:25.890481 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5e9648-8656-4cb6-ab2d-e1ce99e7a7e8 None None] PATH_INFO: `/` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:25.892257 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 350/705] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 26 10:52:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 26 10:52:25.897735 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c18651-e4d7-4a27-bb1f-a6c5601de5ec None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:25.898039 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c18651-e4d7-4a27-bb1f-a6c5601de5ec None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:25.898315 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c18651-e4d7-4a27-bb1f-a6c5601de5ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:25.903183 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76729744-304d-4a3d-b8e0-4db7857aa404 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:52:25.903796 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76729744-304d-4a3d-b8e0-4db7857aa404 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:25.904075 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76729744-304d-4a3d-b8e0-4db7857aa404 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:25.904354 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76729744-304d-4a3d-b8e0-4db7857aa404 None admin] PATH_INFO: `/v3/users` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:25.905085 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76729744-304d-4a3d-b8e0-4db7857aa404 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 10:52:25.905388 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76729744-304d-4a3d-b8e0-4db7857aa404 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:52:25.907110 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76729744-304d-4a3d-b8e0-4db7857aa404 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:52:25.920579 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-64c18651-e4d7-4a27-bb1f-a6c5601de5ec None None] MFA Rules not processed for user `ca155a48e2024c32af3fd2e19e5fac11`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 10:52:25.922063 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-64c18651-e4d7-4a27-bb1f-a6c5601de5ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 10:52:25.929315 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 356/706] 10.222.0.40 () {56 vars in 1133 bytes} [Fri Aug 26 10:52:25 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 10:52:25.935558 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76729744-304d-4a3d-b8e0-4db7857aa404 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:52:26.006055 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 351/707] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 26 10:52:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 26 10:52:26.066573 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76729744-304d-4a3d-b8e0-4db7857aa404 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:52:26.101968 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0dc519-6ca3-4ca3-8479-21ff607d5e17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:52:26.102603 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0dc519-6ca3-4ca3-8479-21ff607d5e17 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:26.102878 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0dc519-6ca3-4ca3-8479-21ff607d5e17 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:26.103145 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0dc519-6ca3-4ca3-8479-21ff607d5e17 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:26.130693 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:52:26.172343 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb0dc519-6ca3-4ca3-8479-21ff607d5e17 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:52:26.173886 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb0dc519-6ca3-4ca3-8479-21ff607d5e17 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:52:26.181841 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c18651-e4d7-4a27-bb1f-a6c5601de5ec None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:52:26.302819 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:52:26.307534 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c18651-e4d7-4a27-bb1f-a6c5601de5ec None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:52:26.347780 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 357/708] 10.222.0.40 () {58 vars in 1282 bytes} [Fri Aug 26 10:52:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 412 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 26 10:52:26.353903 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f632e2a-252e-48fd-b0b5-022390800355 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:52:26.354601 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f632e2a-252e-48fd-b0b5-022390800355 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:26.355006 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0dc519-6ca3-4ca3-8479-21ff607d5e17 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:52:26.355118 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f632e2a-252e-48fd-b0b5-022390800355 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:26.355374 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f632e2a-252e-48fd-b0b5-022390800355 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:26.355964 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f632e2a-252e-48fd-b0b5-022390800355 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:52:26.357331 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f632e2a-252e-48fd-b0b5-022390800355 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:52:26.373281 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application [None req-4f632e2a-252e-48fd-b0b5-022390800355 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 26 10:52:26.373281 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 10:52:26.373281 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 26 10:52:26.373281 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 26 10:52:26.373281 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 26 10:52:26.373281 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 26 10:52:26.373281 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 26 10:52:26.373281 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 26 10:52:26.373281 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 26 10:52:26.373281 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Aug 26 10:52:26.373281 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 26 10:52:26.373281 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Aug 26 10:52:26.373281 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 10:52:26.373281 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 10:52:26.373281 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 10:52:26.373281 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 10:52:26.373281 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 10:52:26.374429 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 10:52:26.374429 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 10:52:26.374429 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 10:52:26.374429 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 10:52:26.374429 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 10:52:26.374429 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 10:52:26.374429 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 26 10:52:26.374429 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 26 10:52:26.374429 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 26 10:52:26.374429 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 26 10:52:26.374429 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 10:52:26.374429 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 10:52:26.374429 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 10:52:26.374429 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 10:52:26.374429 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 10:52:26.374429 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 10:52:26.375476 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 10:52:26.375476 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application with Lock( Aug 26 10:52:26.375476 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 10:52:26.375476 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._enter() Aug 26 10:52:26.375476 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 10:52:26.375476 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 10:52:26.375476 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 10:52:26.375476 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.creator() Aug 26 10:52:26.375476 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 10:52:26.375476 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application created_value = creator( Aug 26 10:52:26.375476 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 26 10:52:26.375476 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 26 10:52:26.375476 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 26 10:52:26.375476 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Aug 26 10:52:26.378978 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 352/709] 10.222.0.40 () {56 vars in 1129 bytes} [Fri Aug 26 10:52:26 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 26 10:52:26.380057 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f632e2a-252e-48fd-b0b5-022390800355 None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:52:26.426154 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:52:26.472593 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0dc519-6ca3-4ca3-8479-21ff607d5e17 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:52:26.501723 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f632e2a-252e-48fd-b0b5-022390800355 None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:52:26.516058 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0dfa3ad-bfe7-4a82-ad6c-5dba37efe0f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:52:26.517257 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0dfa3ad-bfe7-4a82-ad6c-5dba37efe0f8 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:26.518011 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0dfa3ad-bfe7-4a82-ad6c-5dba37efe0f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:26.518308 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0dfa3ad-bfe7-4a82-ad6c-5dba37efe0f8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:26.532466 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0dfa3ad-bfe7-4a82-ad6c-5dba37efe0f8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:52:26.533936 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0dfa3ad-bfe7-4a82-ad6c-5dba37efe0f8 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:52:26.551033 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application [None req-e0dfa3ad-bfe7-4a82-ad6c-5dba37efe0f8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 10:52:26.551033 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 10:52:26.551033 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 10:52:26.551033 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 10:52:26.551033 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 10:52:26.551033 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 10:52:26.551033 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 10:52:26.551033 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 10:52:26.551033 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 10:52:26.551033 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 10:52:26.551033 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 10:52:26.551033 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 10:52:26.551033 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 10:52:26.551033 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 10:52:26.551033 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 10:52:26.551033 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 10:52:26.552044 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 10:52:26.552044 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 10:52:26.552044 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 10:52:26.552044 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 10:52:26.552044 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 10:52:26.552044 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 10:52:26.552044 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 10:52:26.552044 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application with Lock( Aug 26 10:52:26.552044 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 10:52:26.552044 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._enter() Aug 26 10:52:26.552044 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 10:52:26.552044 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 10:52:26.552044 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 10:52:26.552044 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.creator() Aug 26 10:52:26.552044 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 10:52:26.552044 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application created_value = creator( Aug 26 10:52:26.552044 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 10:52:26.552945 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 10:52:26.552945 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 10:52:26.552945 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Aug 26 10:52:26.554587 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad40cf71-bfbc-46dd-a51c-5ad7640c031f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:52:26.555803 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad40cf71-bfbc-46dd-a51c-5ad7640c031f None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:26.556307 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad40cf71-bfbc-46dd-a51c-5ad7640c031f None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:26.556734 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 358/710] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Aug 26 10:52:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 10:52:26.556999 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad40cf71-bfbc-46dd-a51c-5ad7640c031f None admin] PATH_INFO: `/v3/services/network` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:26.557573 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0dfa3ad-bfe7-4a82-ad6c-5dba37efe0f8 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:52:26.558028 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad40cf71-bfbc-46dd-a51c-5ad7640c031f None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:52:26.559627 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad40cf71-bfbc-46dd-a51c-5ad7640c031f None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:52:26.572774 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application [None req-ad40cf71-bfbc-46dd-a51c-5ad7640c031f None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 26 10:52:26.572774 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 10:52:26.572774 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 26 10:52:26.572774 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 26 10:52:26.572774 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 26 10:52:26.572774 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 26 10:52:26.572774 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 26 10:52:26.572774 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 26 10:52:26.572774 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 26 10:52:26.572774 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Aug 26 10:52:26.572774 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 26 10:52:26.572774 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Aug 26 10:52:26.572774 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 10:52:26.572774 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 10:52:26.572774 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 10:52:26.572774 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 10:52:26.572774 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 10:52:26.573806 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 10:52:26.573806 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 10:52:26.573806 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 10:52:26.573806 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 10:52:26.573806 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 10:52:26.573806 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 10:52:26.573806 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 26 10:52:26.573806 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 26 10:52:26.573806 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 26 10:52:26.573806 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 26 10:52:26.573806 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 10:52:26.573806 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 10:52:26.573806 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 10:52:26.573806 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 10:52:26.573806 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 10:52:26.573806 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 10:52:26.574680 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 10:52:26.574680 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application with Lock( Aug 26 10:52:26.574680 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 10:52:26.574680 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._enter() Aug 26 10:52:26.574680 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 10:52:26.574680 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 10:52:26.574680 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 10:52:26.574680 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.creator() Aug 26 10:52:26.574680 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 10:52:26.574680 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application created_value = creator( Aug 26 10:52:26.574680 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 26 10:52:26.574680 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 26 10:52:26.574680 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 26 10:52:26.574680 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Aug 26 10:52:26.577122 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 353/711] 10.222.0.40 () {56 vars in 1111 bytes} [Fri Aug 26 10:52:26 2022] GET /identity/v3/services/network => generated 88 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 10:52:26.577122 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad40cf71-bfbc-46dd-a51c-5ad7640c031f None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:52:26.612874 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:52:26.698550 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0dfa3ad-bfe7-4a82-ad6c-5dba37efe0f8 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:52:26.711607 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:52:26.715677 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad40cf71-bfbc-46dd-a51c-5ad7640c031f None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:52:26.740605 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01c4a3b-0656-454a-af01-900574a52f65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:52:26.741324 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01c4a3b-0656-454a-af01-900574a52f65 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:26.741686 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01c4a3b-0656-454a-af01-900574a52f65 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:26.742154 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01c4a3b-0656-454a-af01-900574a52f65 None admin] PATH_INFO: `/v3/services` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:26.742853 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e01c4a3b-0656-454a-af01-900574a52f65 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 10:52:26.743218 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e01c4a3b-0656-454a-af01-900574a52f65 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:52:26.744673 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e01c4a3b-0656-454a-af01-900574a52f65 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:52:26.753914 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 359/712] 10.222.0.40 () {56 vars in 1124 bytes} [Fri Aug 26 10:52:26 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 10:52:26.753914 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01c4a3b-0656-454a-af01-900574a52f65 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:52:26.769412 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ff71c0-7f06-4abc-8983-85586951b951 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:52:26.770568 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ff71c0-7f06-4abc-8983-85586951b951 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:26.771006 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ff71c0-7f06-4abc-8983-85586951b951 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:26.771700 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ff71c0-7f06-4abc-8983-85586951b951 None admin] PATH_INFO: `/v3/domains` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:26.772879 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ff71c0-7f06-4abc-8983-85586951b951 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 10:52:26.773343 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ff71c0-7f06-4abc-8983-85586951b951 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:52:26.774889 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ff71c0-7f06-4abc-8983-85586951b951 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:52:26.792221 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 354/713] 10.222.0.40 () {56 vars in 1109 bytes} [Fri Aug 26 10:52:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 10:52:26.793210 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ff71c0-7f06-4abc-8983-85586951b951 None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:52:26.876743 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01c4a3b-0656-454a-af01-900574a52f65 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:52:26.905188 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:52:26.921432 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-076f81c5-ec65-4287-bc2b-473f47856fde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:52:26.922346 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076f81c5-ec65-4287-bc2b-473f47856fde None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:26.922630 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076f81c5-ec65-4287-bc2b-473f47856fde None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:26.922937 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076f81c5-ec65-4287-bc2b-473f47856fde None admin] PATH_INFO: `/v3/services` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:26.923734 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-076f81c5-ec65-4287-bc2b-473f47856fde None admin] RBAC: Adding query filter params (name=network) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 10:52:26.924040 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-076f81c5-ec65-4287-bc2b-473f47856fde None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:52:26.925317 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ff71c0-7f06-4abc-8983-85586951b951 None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:52:26.925428 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-076f81c5-ec65-4287-bc2b-473f47856fde None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:52:26.935603 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 360/714] 10.222.0.40 () {56 vars in 1112 bytes} [Fri Aug 26 10:52:26 2022] GET /identity/v3/services?name=network => generated 124 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 10:52:26.935603 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-076f81c5-ec65-4287-bc2b-473f47856fde None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:52:26.966492 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20477127-1af8-4fe7-a706-3ae6907d5359 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:52:26.967546 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20477127-1af8-4fe7-a706-3ae6907d5359 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:26.968080 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20477127-1af8-4fe7-a706-3ae6907d5359 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:26.968634 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20477127-1af8-4fe7-a706-3ae6907d5359 None admin] PATH_INFO: `/v3/services` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:26.969483 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20477127-1af8-4fe7-a706-3ae6907d5359 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 10:52:26.970558 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20477127-1af8-4fe7-a706-3ae6907d5359 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:52:26.972485 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20477127-1af8-4fe7-a706-3ae6907d5359 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:52:26.979820 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 355/715] 10.222.0.40 () {56 vars in 1124 bytes} [Fri Aug 26 10:52:26 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 10:52:26.981016 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20477127-1af8-4fe7-a706-3ae6907d5359 None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:52:26.996995 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:52:27.109845 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-076f81c5-ec65-4287-bc2b-473f47856fde None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:52:27.147819 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20477127-1af8-4fe7-a706-3ae6907d5359 None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:52:27.148177 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704817f9-7a5f-4926-85fe-07b073cf2d5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:52:27.148796 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704817f9-7a5f-4926-85fe-07b073cf2d5c None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:27.149069 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704817f9-7a5f-4926-85fe-07b073cf2d5c None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:27.149336 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704817f9-7a5f-4926-85fe-07b073cf2d5c None admin] PATH_INFO: `/v3/projects/service` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:27.156891 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-704817f9-7a5f-4926-85fe-07b073cf2d5c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:52:27.158632 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-704817f9-7a5f-4926-85fe-07b073cf2d5c None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:52:27.170313 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application [None req-704817f9-7a5f-4926-85fe-07b073cf2d5c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 10:52:27.170313 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 10:52:27.170313 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 10:52:27.170313 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 10:52:27.170313 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 10:52:27.170313 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 10:52:27.170313 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 10:52:27.170313 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 10:52:27.170313 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 10:52:27.170313 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 10:52:27.170313 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 10:52:27.170313 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 10:52:27.170313 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 26 10:52:27.170313 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 26 10:52:27.170313 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 26 10:52:27.171244 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 26 10:52:27.171244 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 10:52:27.171244 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 10:52:27.171244 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 10:52:27.171244 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 10:52:27.171244 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 10:52:27.171244 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 10:52:27.171244 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 10:52:27.171244 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application with Lock( Aug 26 10:52:27.171244 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 10:52:27.171244 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._enter() Aug 26 10:52:27.171244 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 10:52:27.171244 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 10:52:27.171244 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 10:52:27.171244 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.creator() Aug 26 10:52:27.171244 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 10:52:27.171244 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application created_value = creator( Aug 26 10:52:27.172803 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 26 10:52:27.172803 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 26 10:52:27.172803 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 26 10:52:27.172803 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 26 10:52:27.172803 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 26 10:52:27.172803 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 26 10:52:27.172803 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 10:52:27.172803 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Aug 26 10:52:27.172803 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 361/716] 10.222.0.40 () {56 vars in 1111 bytes} [Fri Aug 26 10:52:26 2022] GET /identity/v3/projects/service => generated 88 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 10:52:27.177439 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704817f9-7a5f-4926-85fe-07b073cf2d5c None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:52:27.198864 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3d41fb-3977-4b60-b1d3-b07d4163dd1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:52:27.199895 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3d41fb-3977-4b60-b1d3-b07d4163dd1d None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:27.200374 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3d41fb-3977-4b60-b1d3-b07d4163dd1d None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:27.200885 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3d41fb-3977-4b60-b1d3-b07d4163dd1d None admin] PATH_INFO: `/v3/services` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:27.201765 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be3d41fb-3977-4b60-b1d3-b07d4163dd1d None admin] RBAC: Adding query filter params (type=network) {{(pid=81423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 10:52:27.202295 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be3d41fb-3977-4b60-b1d3-b07d4163dd1d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:52:27.204010 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be3d41fb-3977-4b60-b1d3-b07d4163dd1d None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:52:27.219879 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 356/717] 10.222.0.40 () {56 vars in 1112 bytes} [Fri Aug 26 10:52:26 2022] GET /identity/v3/services?type=network => generated 124 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 10:52:27.294263 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704817f9-7a5f-4926-85fe-07b073cf2d5c None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:52:27.297465 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:52:27.332949 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09e96c5-49fc-409a-9a4b-475166bf23d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:52:27.333567 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09e96c5-49fc-409a-9a4b-475166bf23d5 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:27.333841 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09e96c5-49fc-409a-9a4b-475166bf23d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:27.334139 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09e96c5-49fc-409a-9a4b-475166bf23d5 None admin] PATH_INFO: `/v3/projects` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:27.334839 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b09e96c5-49fc-409a-9a4b-475166bf23d5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 10:52:27.335137 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b09e96c5-49fc-409a-9a4b-475166bf23d5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:52:27.336586 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b09e96c5-49fc-409a-9a4b-475166bf23d5 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:52:27.349871 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 362/718] 10.222.0.40 () {56 vars in 1148 bytes} [Fri Aug 26 10:52:27 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 10:52:27.359370 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3d41fb-3977-4b60-b1d3-b07d4163dd1d None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:52:27.391155 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:52:27.496761 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3d41fb-3977-4b60-b1d3-b07d4163dd1d None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:52:27.533413 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae25a704-f759-48a4-a502-db77fb0abbc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:52:27.534058 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae25a704-f759-48a4-a502-db77fb0abbc9 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:27.534338 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae25a704-f759-48a4-a502-db77fb0abbc9 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:27.534605 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae25a704-f759-48a4-a502-db77fb0abbc9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:27.535331 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae25a704-f759-48a4-a502-db77fb0abbc9 None admin] RBAC: Adding query filter params (role.id=1f97e4602e2e4d6498913a81f9ae8d6a, scope.project.id=0e30ccf0c1204727b786876c141dbcd2, user.id=1f4b4a662db049ecacc865ecff10bdb0) {{(pid=81423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 10:52:27.535632 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae25a704-f759-48a4-a502-db77fb0abbc9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:52:27.537201 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae25a704-f759-48a4-a502-db77fb0abbc9 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:52:27.543388 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 357/719] 10.222.0.40 () {56 vars in 1374 bytes} [Fri Aug 26 10:52:27 2022] GET /identity/v3/role_assignments?user.id=1f4b4a662db049ecacc865ecff10bdb0&scope.project.id=0e30ccf0c1204727b786876c141dbcd2&role.id=1f97e4602e2e4d6498913a81f9ae8d6a => generated 259 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 10:52:27.659041 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95058ecc-20fc-4f08-b445-b139fa1bfae6 None None] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:27.659385 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95058ecc-20fc-4f08-b445-b139fa1bfae6 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:27.659595 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95058ecc-20fc-4f08-b445-b139fa1bfae6 None None] PATH_INFO: `/` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:27.661317 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 363/720] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 26 10:52:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 10:52:27.666817 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf4e563-7256-47f6-b034-e8caff4c0044 None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:27.667285 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf4e563-7256-47f6-b034-e8caff4c0044 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:27.667359 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf4e563-7256-47f6-b034-e8caff4c0044 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:27.695723 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-6bf4e563-7256-47f6-b034-e8caff4c0044 None None] MFA Rules not processed for user `ca155a48e2024c32af3fd2e19e5fac11`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 10:52:27.696969 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-6bf4e563-7256-47f6-b034-e8caff4c0044 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 10:52:27.793715 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 358/721] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 26 10:52:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 10:52:28.075232 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95058ecc-20fc-4f08-b445-b139fa1bfae6 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:52:28.161662 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b06ee37-a47c-46c2-b823-dce4470f8100 None None] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:28.162374 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b06ee37-a47c-46c2-b823-dce4470f8100 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:28.162973 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b06ee37-a47c-46c2-b823-dce4470f8100 None None] PATH_INFO: `/` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:28.165390 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 359/722] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 26 10:52:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 10:52:28.169482 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2070ca-2e4a-4156-9654-37f5703ce1a9 None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:28.169833 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2070ca-2e4a-4156-9654-37f5703ce1a9 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:28.170202 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2070ca-2e4a-4156-9654-37f5703ce1a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:28.197196 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-1d2070ca-2e4a-4156-9654-37f5703ce1a9 None None] MFA Rules not processed for user `ca155a48e2024c32af3fd2e19e5fac11`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 10:52:28.198416 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-1d2070ca-2e4a-4156-9654-37f5703ce1a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 10:52:28.206303 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:52:28.214274 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95058ecc-20fc-4f08-b445-b139fa1bfae6 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:52:28.285044 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0823afce-ab3f-4ae7-b0fc-abc4d3d9a3d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:52:28.285678 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0823afce-ab3f-4ae7-b0fc-abc4d3d9a3d6 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:28.285952 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0823afce-ab3f-4ae7-b0fc-abc4d3d9a3d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:28.286262 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0823afce-ab3f-4ae7-b0fc-abc4d3d9a3d6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:28.309320 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0823afce-ab3f-4ae7-b0fc-abc4d3d9a3d6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 10:52:28.314783 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application [None req-0823afce-ab3f-4ae7-b0fc-abc4d3d9a3d6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 26 10:52:28.314783 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 10:52:28.314783 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 10:52:28.314783 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 10:52:28.314783 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 10:52:28.314783 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 10:52:28.314783 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 10:52:28.314783 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 10:52:28.314783 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 10:52:28.314783 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 10:52:28.314783 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 10:52:28.314783 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 10:52:28.314783 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 10:52:28.314783 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 10:52:28.314783 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 10:52:28.314783 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 10:52:28.315736 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 10:52:28.315736 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 10:52:28.315736 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 10:52:28.315736 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 10:52:28.315736 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 10:52:28.315736 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 10:52:28.315736 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 10:52:28.315736 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 10:52:28.315736 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 10:52:28.315736 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 10:52:28.315736 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 10:52:28.315736 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application with Lock( Aug 26 10:52:28.315736 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 10:52:28.315736 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._enter() Aug 26 10:52:28.315736 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 10:52:28.315736 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 10:52:28.316613 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 10:52:28.316613 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.creator() Aug 26 10:52:28.316613 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 10:52:28.316613 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application created_value = creator( Aug 26 10:52:28.316613 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 10:52:28.316613 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 10:52:28.316613 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 10:52:28.316613 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 10:52:28.316613 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 10:52:28.316613 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 10:52:28.316613 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 26 10:52:28.316613 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Aug 26 10:52:28.317279 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 364/723] 10.222.0.40 () {56 vars in 1096 bytes} [Fri Aug 26 10:52:28 2022] GET /identity/v3/roles/admin => generated 83 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 10:52:28.324679 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0823afce-ab3f-4ae7-b0fc-abc4d3d9a3d6 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:52:28.340056 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 360/724] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 26 10:52:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 26 10:52:28.359811 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:52:28.476189 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0823afce-ab3f-4ae7-b0fc-abc4d3d9a3d6 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:52:28.524006 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4ca9a4-0217-4b02-85aa-15e77e3b1795 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:52:28.524608 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4ca9a4-0217-4b02-85aa-15e77e3b1795 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:28.524885 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4ca9a4-0217-4b02-85aa-15e77e3b1795 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:28.525155 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4ca9a4-0217-4b02-85aa-15e77e3b1795 None admin] PATH_INFO: `/v3/roles` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:28.525811 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e4ca9a4-0217-4b02-85aa-15e77e3b1795 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 10:52:28.526134 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e4ca9a4-0217-4b02-85aa-15e77e3b1795 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:52:28.527493 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e4ca9a4-0217-4b02-85aa-15e77e3b1795 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:52:28.536498 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 365/725] 10.222.0.40 () {56 vars in 1099 bytes} [Fri Aug 26 10:52:28 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 10:52:28.541713 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2070ca-2e4a-4156-9654-37f5703ce1a9 None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:52:28.731737 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2070ca-2e4a-4156-9654-37f5703ce1a9 None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:52:28.742247 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4ca9a4-0217-4b02-85aa-15e77e3b1795 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:52:28.771054 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42fd4d3-61a0-4fcb-b056-24834a01154b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:52:28.772033 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42fd4d3-61a0-4fcb-b056-24834a01154b None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:28.772637 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42fd4d3-61a0-4fcb-b056-24834a01154b None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:28.773111 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42fd4d3-61a0-4fcb-b056-24834a01154b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:28.780034 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:52:28.787064 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b42fd4d3-61a0-4fcb-b056-24834a01154b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:52:28.788748 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b42fd4d3-61a0-4fcb-b056-24834a01154b None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:52:28.811024 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application [None req-b42fd4d3-61a0-4fcb-b056-24834a01154b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 10:52:28.811024 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 10:52:28.811024 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 10:52:28.811024 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 10:52:28.811024 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 10:52:28.811024 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 10:52:28.811024 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 10:52:28.811024 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 10:52:28.811024 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 10:52:28.811024 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 10:52:28.811024 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 10:52:28.811024 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 10:52:28.811024 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 10:52:28.811024 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 10:52:28.811024 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 10:52:28.811024 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 10:52:28.812042 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 10:52:28.812042 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 10:52:28.812042 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 10:52:28.812042 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 10:52:28.812042 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 10:52:28.812042 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 10:52:28.812042 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 10:52:28.812042 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application with Lock( Aug 26 10:52:28.812042 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 10:52:28.812042 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._enter() Aug 26 10:52:28.812042 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 10:52:28.812042 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 10:52:28.812042 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 10:52:28.812042 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.creator() Aug 26 10:52:28.812042 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 10:52:28.812042 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application created_value = creator( Aug 26 10:52:28.812042 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 10:52:28.812946 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 10:52:28.812946 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 10:52:28.812946 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Aug 26 10:52:28.824757 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 361/726] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Aug 26 10:52:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 283 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 10:52:28.826382 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42fd4d3-61a0-4fcb-b056-24834a01154b None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:52:28.930290 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:52:28.961323 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42fd4d3-61a0-4fcb-b056-24834a01154b None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:52:28.975085 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4ca9a4-0217-4b02-85aa-15e77e3b1795 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:52:29.031559 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea8fa26-6c45-44e7-895f-6c98cbc74337 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:52:29.032688 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea8fa26-6c45-44e7-895f-6c98cbc74337 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:29.033043 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea8fa26-6c45-44e7-895f-6c98cbc74337 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:29.033311 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea8fa26-6c45-44e7-895f-6c98cbc74337 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:29.034453 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3efe6f-3bb4-4da6-a5b2-935540bf9236 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:52:29.035156 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3efe6f-3bb4-4da6-a5b2-935540bf9236 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:29.035422 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3efe6f-3bb4-4da6-a5b2-935540bf9236 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:29.035685 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3efe6f-3bb4-4da6-a5b2-935540bf9236 None admin] PATH_INFO: `/v3/domains` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:29.036328 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc3efe6f-3bb4-4da6-a5b2-935540bf9236 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 10:52:29.036622 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc3efe6f-3bb4-4da6-a5b2-935540bf9236 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:52:29.037993 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc3efe6f-3bb4-4da6-a5b2-935540bf9236 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:52:29.055796 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea8fa26-6c45-44e7-895f-6c98cbc74337 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 10:52:29.061016 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application [None req-0ea8fa26-6c45-44e7-895f-6c98cbc74337 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 26 10:52:29.061016 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 10:52:29.061016 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 10:52:29.061016 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 10:52:29.061016 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 10:52:29.061016 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 10:52:29.061016 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 10:52:29.061016 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 10:52:29.061016 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 10:52:29.061016 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 10:52:29.061016 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 10:52:29.061016 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 10:52:29.061016 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 10:52:29.061016 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 10:52:29.061016 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 10:52:29.061016 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 10:52:29.061954 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 10:52:29.061954 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 10:52:29.061954 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 10:52:29.061954 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 10:52:29.061954 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 10:52:29.061954 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 10:52:29.061954 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 10:52:29.061954 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 10:52:29.061954 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 10:52:29.061954 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 10:52:29.061954 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 10:52:29.061954 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application with Lock( Aug 26 10:52:29.061954 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 10:52:29.061954 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._enter() Aug 26 10:52:29.061954 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 10:52:29.061954 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 10:52:29.062841 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 10:52:29.062841 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.creator() Aug 26 10:52:29.062841 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 10:52:29.062841 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application created_value = creator( Aug 26 10:52:29.062841 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 10:52:29.062841 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 10:52:29.062841 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 10:52:29.062841 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 10:52:29.062841 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 10:52:29.062841 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 10:52:29.062841 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 26 10:52:29.062841 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Aug 26 10:52:29.062841 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 366/727] 10.222.0.40 () {56 vars in 1102 bytes} [Fri Aug 26 10:52:28 2022] GET /identity/v3/roles/service => generated 85 bytes in 320 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 10:52:29.068002 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 362/728] 10.222.0.40 () {56 vars in 1109 bytes} [Fri Aug 26 10:52:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 10:52:29.072876 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea8fa26-6c45-44e7-895f-6c98cbc74337 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:52:29.074135 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3efe6f-3bb4-4da6-a5b2-935540bf9236 None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:52:29.189101 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea8fa26-6c45-44e7-895f-6c98cbc74337 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:52:29.207542 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:52:29.228977 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6238d3-4dd7-4b2d-a097-d8106aac668c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:52:29.230549 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6238d3-4dd7-4b2d-a097-d8106aac668c None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:29.231073 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6238d3-4dd7-4b2d-a097-d8106aac668c None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:29.231519 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6238d3-4dd7-4b2d-a097-d8106aac668c None admin] PATH_INFO: `/v3/roles` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:29.232365 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa6238d3-4dd7-4b2d-a097-d8106aac668c None admin] RBAC: Adding query filter params (name=service) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 10:52:29.232808 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa6238d3-4dd7-4b2d-a097-d8106aac668c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:52:29.234967 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa6238d3-4dd7-4b2d-a097-d8106aac668c None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:52:29.245002 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 367/729] 10.222.0.40 () {56 vars in 1103 bytes} [Fri Aug 26 10:52:29 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 10:52:29.250696 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6238d3-4dd7-4b2d-a097-d8106aac668c None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:52:29.266315 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:52:29.329475 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3efe6f-3bb4-4da6-a5b2-935540bf9236 None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:52:29.374197 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b29d25f-8525-4327-9381-e028ae9ab296 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:52:29.374836 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b29d25f-8525-4327-9381-e028ae9ab296 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:29.375117 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b29d25f-8525-4327-9381-e028ae9ab296 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:29.375387 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b29d25f-8525-4327-9381-e028ae9ab296 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:29.383058 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b29d25f-8525-4327-9381-e028ae9ab296 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:52:29.384585 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b29d25f-8525-4327-9381-e028ae9ab296 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:52:29.392307 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6238d3-4dd7-4b2d-a097-d8106aac668c None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:52:29.396666 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application [None req-1b29d25f-8525-4327-9381-e028ae9ab296 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 26 10:52:29.396666 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 10:52:29.396666 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 10:52:29.396666 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 10:52:29.396666 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 10:52:29.396666 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 10:52:29.396666 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 10:52:29.396666 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 10:52:29.396666 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 10:52:29.396666 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 10:52:29.396666 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 10:52:29.396666 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 10:52:29.396666 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 26 10:52:29.396666 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 26 10:52:29.396666 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 26 10:52:29.396666 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 26 10:52:29.397604 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 10:52:29.397604 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 10:52:29.397604 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 26 10:52:29.397604 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 10:52:29.397604 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 26 10:52:29.397604 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 10:52:29.397604 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 10:52:29.397604 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 10:52:29.397604 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 10:52:29.397604 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 10:52:29.397604 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 10:52:29.397604 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application with Lock( Aug 26 10:52:29.397604 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 10:52:29.397604 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._enter() Aug 26 10:52:29.397604 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 10:52:29.397604 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 10:52:29.397604 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 10:52:29.397604 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.creator() Aug 26 10:52:29.398581 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 10:52:29.398581 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application created_value = creator( Aug 26 10:52:29.398581 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 26 10:52:29.398581 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 26 10:52:29.398581 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 26 10:52:29.398581 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 26 10:52:29.398581 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 26 10:52:29.398581 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 26 10:52:29.398581 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 26 10:52:29.398581 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Aug 26 10:52:29.398581 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 363/730] 10.222.0.40 () {56 vars in 1096 bytes} [Fri Aug 26 10:52:29 2022] GET /identity/v3/users/swift => generated 83 bytes in 324 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 10:52:29.411075 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b29d25f-8525-4327-9381-e028ae9ab296 None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:52:29.434460 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c2abb5-69e9-4d10-a5e0-a4524cd00087 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:52:29.435104 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c2abb5-69e9-4d10-a5e0-a4524cd00087 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:29.435380 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c2abb5-69e9-4d10-a5e0-a4524cd00087 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:29.435649 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c2abb5-69e9-4d10-a5e0-a4524cd00087 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:29.447776 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c2abb5-69e9-4d10-a5e0-a4524cd00087 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:52:29.449379 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c2abb5-69e9-4d10-a5e0-a4524cd00087 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:52:29.465883 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application [None req-92c2abb5-69e9-4d10-a5e0-a4524cd00087 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 10:52:29.465883 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 10:52:29.465883 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 10:52:29.465883 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 10:52:29.465883 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 10:52:29.465883 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 10:52:29.465883 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 10:52:29.465883 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 10:52:29.465883 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 10:52:29.465883 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 10:52:29.465883 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 10:52:29.465883 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 10:52:29.465883 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 10:52:29.465883 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 10:52:29.465883 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 10:52:29.465883 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 10:52:29.468149 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 10:52:29.468149 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 10:52:29.468149 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 10:52:29.468149 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 10:52:29.468149 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 10:52:29.468149 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 10:52:29.468149 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 10:52:29.468149 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application with Lock( Aug 26 10:52:29.468149 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 10:52:29.468149 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._enter() Aug 26 10:52:29.468149 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 10:52:29.468149 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 10:52:29.468149 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 10:52:29.468149 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.creator() Aug 26 10:52:29.468149 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 10:52:29.468149 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application created_value = creator( Aug 26 10:52:29.468149 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 10:52:29.470452 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 10:52:29.470452 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 10:52:29.470452 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Aug 26 10:52:29.470452 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 368/731] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Aug 26 10:52:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 26 10:52:29.470452 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7e8fba-c8a8-4c8c-988d-50b42313ad3f None None] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:29.470452 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7e8fba-c8a8-4c8c-988d-50b42313ad3f None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:29.470452 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7e8fba-c8a8-4c8c-988d-50b42313ad3f None None] PATH_INFO: `/` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:29.474698 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 369/732] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 26 10:52:29 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 10:52:29.474698 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7e8fba-c8a8-4c8c-988d-50b42313ad3f None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:52:29.552507 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b29d25f-8525-4327-9381-e028ae9ab296 None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:52:29.553203 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:52:29.567562 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:52:29.603019 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e102f7a-aa99-4271-a6cd-ebdaa248a1fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:52:29.603548 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e102f7a-aa99-4271-a6cd-ebdaa248a1fe None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:29.603824 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e102f7a-aa99-4271-a6cd-ebdaa248a1fe None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:29.604094 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e102f7a-aa99-4271-a6cd-ebdaa248a1fe None admin] PATH_INFO: `/v3/users` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:29.604824 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e102f7a-aa99-4271-a6cd-ebdaa248a1fe None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 10:52:29.605217 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e102f7a-aa99-4271-a6cd-ebdaa248a1fe None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:52:29.606770 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e102f7a-aa99-4271-a6cd-ebdaa248a1fe None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:52:29.613643 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7e8fba-c8a8-4c8c-988d-50b42313ad3f None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:52:29.626729 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 364/733] 10.222.0.40 () {56 vars in 1135 bytes} [Fri Aug 26 10:52:29 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 10:52:29.628197 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167ab0de-6637-4fca-aaeb-67cf2cbf48bd None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:29.628487 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167ab0de-6637-4fca-aaeb-67cf2cbf48bd None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:29.628852 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167ab0de-6637-4fca-aaeb-67cf2cbf48bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:29.648274 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-167ab0de-6637-4fca-aaeb-67cf2cbf48bd None None] MFA Rules not processed for user `ca155a48e2024c32af3fd2e19e5fac11`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 10:52:29.649465 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-167ab0de-6637-4fca-aaeb-67cf2cbf48bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 10:52:29.651012 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a908498d-e382-443d-8c16-b9483b48b7df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:52:29.651677 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a908498d-e382-443d-8c16-b9483b48b7df None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:29.652007 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a908498d-e382-443d-8c16-b9483b48b7df None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:29.652279 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a908498d-e382-443d-8c16-b9483b48b7df None admin] PATH_INFO: `/v3/domains` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:29.652922 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a908498d-e382-443d-8c16-b9483b48b7df None admin] RBAC: Adding query filter params (name=Default) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 10:52:29.653267 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a908498d-e382-443d-8c16-b9483b48b7df None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:52:29.654723 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a908498d-e382-443d-8c16-b9483b48b7df None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:52:29.672075 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 370/734] 10.222.0.40 () {56 vars in 1109 bytes} [Fri Aug 26 10:52:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 10:52:29.673817 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a908498d-e382-443d-8c16-b9483b48b7df None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:52:29.730105 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 365/735] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 26 10:52:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 10:52:29.731128 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167ab0de-6637-4fca-aaeb-67cf2cbf48bd None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:52:29.778796 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a908498d-e382-443d-8c16-b9483b48b7df None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:52:29.793837 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:52:29.820204 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b63917-49ac-47c5-9b5f-5e8a837b470a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:52:29.821185 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b63917-49ac-47c5-9b5f-5e8a837b470a None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:29.821830 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b63917-49ac-47c5-9b5f-5e8a837b470a None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:29.822349 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b63917-49ac-47c5-9b5f-5e8a837b470a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:29.891163 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:52:29.899113 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167ab0de-6637-4fca-aaeb-67cf2cbf48bd None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:52:29.900616 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b63917-49ac-47c5-9b5f-5e8a837b470a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:52:29.902228 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b63917-49ac-47c5-9b5f-5e8a837b470a None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:52:29.954916 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06de7236-7875-4925-a4d9-ef2efc0a9d9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:52:29.955542 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06de7236-7875-4925-a4d9-ef2efc0a9d9d None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:29.955821 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06de7236-7875-4925-a4d9-ef2efc0a9d9d None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:29.956085 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06de7236-7875-4925-a4d9-ef2efc0a9d9d None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:29.966090 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06de7236-7875-4925-a4d9-ef2efc0a9d9d None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:52:29.967723 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06de7236-7875-4925-a4d9-ef2efc0a9d9d None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:52:29.979221 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application [None req-06de7236-7875-4925-a4d9-ef2efc0a9d9d None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 26 10:52:29.979221 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 10:52:29.979221 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 10:52:29.979221 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 10:52:29.979221 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 10:52:29.979221 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 10:52:29.979221 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 10:52:29.979221 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 10:52:29.979221 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 10:52:29.979221 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 10:52:29.979221 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 10:52:29.979221 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 10:52:29.979221 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 26 10:52:29.979221 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 26 10:52:29.979221 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 26 10:52:29.979221 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 26 10:52:29.980194 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 10:52:29.980194 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 10:52:29.980194 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 26 10:52:29.980194 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 10:52:29.980194 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 26 10:52:29.980194 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 10:52:29.980194 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 10:52:29.980194 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 10:52:29.980194 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 10:52:29.980194 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 10:52:29.980194 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 10:52:29.980194 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application with Lock( Aug 26 10:52:29.980194 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 10:52:29.980194 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._enter() Aug 26 10:52:29.980194 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 10:52:29.980194 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 10:52:29.980194 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 10:52:29.980194 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.creator() Aug 26 10:52:29.981123 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 10:52:29.981123 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application created_value = creator( Aug 26 10:52:29.981123 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 26 10:52:29.981123 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 26 10:52:29.981123 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 26 10:52:29.981123 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 26 10:52:29.981123 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 26 10:52:29.981123 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 26 10:52:29.981123 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 26 10:52:29.981123 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Aug 26 10:52:29.985722 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 366/736] 10.222.0.40 () {56 vars in 1117 bytes} [Fri Aug 26 10:52:29 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 10:52:29.987081 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e307109-7546-4b52-ac67-96f36f80fd28 None None] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:29.987356 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e307109-7546-4b52-ac67-96f36f80fd28 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:29.987642 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e307109-7546-4b52-ac67-96f36f80fd28 None None] PATH_INFO: `/` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:29.988875 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 367/737] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 26 10:52:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 10:52:29.989858 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35cb4ab-d6fe-4cf8-8a57-ab3332fca4d5 None None] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:29.990157 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35cb4ab-d6fe-4cf8-8a57-ab3332fca4d5 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:29.990423 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35cb4ab-d6fe-4cf8-8a57-ab3332fca4d5 None None] PATH_INFO: `/` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:29.994221 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 368/738] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 26 10:52:29 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 10:52:29.994221 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35cb4ab-d6fe-4cf8-8a57-ab3332fca4d5 None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:52:30.022575 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 371/739] 10.222.0.40 () {58 vars in 1282 bytes} [Fri Aug 26 10:52:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 26 10:52:30.022575 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b92302-307d-45b4-8132-48ec4f77d458 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:30.022575 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b92302-307d-45b4-8132-48ec4f77d458 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:30.022575 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b92302-307d-45b4-8132-48ec4f77d458 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:30.037876 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-f7b92302-307d-45b4-8132-48ec4f77d458 None None] MFA Rules not processed for user `ca155a48e2024c32af3fd2e19e5fac11`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 10:52:30.039080 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-f7b92302-307d-45b4-8132-48ec4f77d458 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 10:52:30.048761 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:52:30.105438 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 372/740] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 26 10:52:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 10:52:30.106702 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5bc872-59e9-4f78-aa6b-8b6e59505610 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:30.106977 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5bc872-59e9-4f78-aa6b-8b6e59505610 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:30.107250 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5bc872-59e9-4f78-aa6b-8b6e59505610 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:30.113585 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35cb4ab-d6fe-4cf8-8a57-ab3332fca4d5 None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:52:30.129732 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-8d5bc872-59e9-4f78-aa6b-8b6e59505610 None None] MFA Rules not processed for user `ca155a48e2024c32af3fd2e19e5fac11`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 10:52:30.130845 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-8d5bc872-59e9-4f78-aa6b-8b6e59505610 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 10:52:30.151421 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8a805d-7f6a-4544-b784-9bc6fddc78c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:52:30.152025 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8a805d-7f6a-4544-b784-9bc6fddc78c4 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:30.152301 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8a805d-7f6a-4544-b784-9bc6fddc78c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:30.152571 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8a805d-7f6a-4544-b784-9bc6fddc78c4 None admin] PATH_INFO: `/v3/users` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:30.153302 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df8a805d-7f6a-4544-b784-9bc6fddc78c4 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 10:52:30.153605 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df8a805d-7f6a-4544-b784-9bc6fddc78c4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:52:30.155062 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df8a805d-7f6a-4544-b784-9bc6fddc78c4 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:52:30.178165 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 369/741] 10.222.0.40 () {56 vars in 1149 bytes} [Fri Aug 26 10:52:29 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 10:52:30.178165 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8a805d-7f6a-4544-b784-9bc6fddc78c4 None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:52:30.194113 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:52:30.207092 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 373/742] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 26 10:52:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 10:52:30.207979 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5bc872-59e9-4f78-aa6b-8b6e59505610 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:52:30.304105 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:52:30.307058 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8a805d-7f6a-4544-b784-9bc6fddc78c4 None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:52:30.341912 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5bc872-59e9-4f78-aa6b-8b6e59505610 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:52:30.352474 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83ccee9-f98b-488e-ae48-d34266ea330b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:52:30.353326 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83ccee9-f98b-488e-ae48-d34266ea330b None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:30.353873 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83ccee9-f98b-488e-ae48-d34266ea330b None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:30.354379 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83ccee9-f98b-488e-ae48-d34266ea330b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:30.368051 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e83ccee9-f98b-488e-ae48-d34266ea330b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:52:30.369793 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e83ccee9-f98b-488e-ae48-d34266ea330b None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:52:30.384437 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533e604c-bb1e-4100-9af2-f4c1b4677d56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:52:30.385061 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533e604c-bb1e-4100-9af2-f4c1b4677d56 None admin] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:30.385413 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533e604c-bb1e-4100-9af2-f4c1b4677d56 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:30.385755 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533e604c-bb1e-4100-9af2-f4c1b4677d56 None admin] PATH_INFO: `/v3/services` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:30.386490 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application [None req-e83ccee9-f98b-488e-ae48-d34266ea330b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 10:52:30.386490 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 10:52:30.386490 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 10:52:30.386490 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 10:52:30.386490 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 10:52:30.386490 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 10:52:30.386490 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 10:52:30.386490 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 10:52:30.386490 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 10:52:30.386490 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 10:52:30.386490 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 10:52:30.386490 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 10:52:30.386490 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 10:52:30.386490 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 10:52:30.386490 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 10:52:30.386490 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 10:52:30.387388 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 10:52:30.387388 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 10:52:30.387388 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 10:52:30.387388 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 10:52:30.387388 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 10:52:30.387388 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 10:52:30.387388 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 10:52:30.387388 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application with Lock( Aug 26 10:52:30.387388 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 10:52:30.387388 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._enter() Aug 26 10:52:30.387388 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 10:52:30.387388 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 10:52:30.387388 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 10:52:30.387388 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.creator() Aug 26 10:52:30.387388 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 10:52:30.387388 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application created_value = creator( Aug 26 10:52:30.387388 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 10:52:30.388304 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 10:52:30.388304 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 10:52:30.388304 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Aug 26 10:52:30.388471 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-533e604c-bb1e-4100-9af2-f4c1b4677d56 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:52:30.388471 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-533e604c-bb1e-4100-9af2-f4c1b4677d56 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:52:30.390634 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 370/743] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Aug 26 10:52:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 10:52:30.391673 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83ccee9-f98b-488e-ae48-d34266ea330b None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:52:30.409133 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 374/744] 10.222.0.40 () {60 vars in 1141 bytes} [Fri Aug 26 10:52:30 2022] POST /identity/v3/services => generated 251 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 10:52:30.410164 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533e604c-bb1e-4100-9af2-f4c1b4677d56 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:52:30.480656 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:52:30.521089 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83ccee9-f98b-488e-ae48-d34266ea330b None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:52:30.543438 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533e604c-bb1e-4100-9af2-f4c1b4677d56 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:52:30.560629 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1499b6-a9f8-442c-82f5-cf4426720ee5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:52:30.561250 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1499b6-a9f8-442c-82f5-cf4426720ee5 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:30.561534 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1499b6-a9f8-442c-82f5-cf4426720ee5 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:30.561805 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1499b6-a9f8-442c-82f5-cf4426720ee5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:30.580933 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:52:30.588298 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd426b22-c875-4194-8a11-a4d478c2ab47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:52:30.588482 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd426b22-c875-4194-8a11-a4d478c2ab47 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:30.588669 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd426b22-c875-4194-8a11-a4d478c2ab47 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:30.588883 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd426b22-c875-4194-8a11-a4d478c2ab47 None admin] PATH_INFO: `/v3/domains` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:30.589538 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd426b22-c875-4194-8a11-a4d478c2ab47 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 10:52:30.589840 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd426b22-c875-4194-8a11-a4d478c2ab47 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:52:30.591329 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd426b22-c875-4194-8a11-a4d478c2ab47 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:52:30.604498 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 375/745] 10.222.0.40 () {56 vars in 1109 bytes} [Fri Aug 26 10:52:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 10:52:30.605442 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd426b22-c875-4194-8a11-a4d478c2ab47 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:52:30.644619 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a1499b6-a9f8-442c-82f5-cf4426720ee5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:52:30.646020 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a1499b6-a9f8-442c-82f5-cf4426720ee5 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:52:30.713874 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:52:30.722776 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd426b22-c875-4194-8a11-a4d478c2ab47 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:52:30.751461 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 371/746] 10.222.0.40 () {58 vars in 1282 bytes} [Fri Aug 26 10:52:30 2022] GET /identity/v3/auth/tokens => generated 903 bytes in 360 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 26 10:52:30.752699 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1499b6-a9f8-442c-82f5-cf4426720ee5 None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:52:30.767196 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde5dcd7-0c56-435e-b222-f1198d01b927 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:52:30.767818 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde5dcd7-0c56-435e-b222-f1198d01b927 None admin] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:30.768095 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde5dcd7-0c56-435e-b222-f1198d01b927 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:30.768364 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde5dcd7-0c56-435e-b222-f1198d01b927 None admin] PATH_INFO: `/v3/services` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:30.769005 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde5dcd7-0c56-435e-b222-f1198d01b927 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:52:30.770459 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde5dcd7-0c56-435e-b222-f1198d01b927 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:52:30.786620 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 376/747] 10.222.0.40 () {60 vars in 1141 bytes} [Fri Aug 26 10:52:30 2022] POST /identity/v3/services => generated 240 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 10:52:30.795163 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde5dcd7-0c56-435e-b222-f1198d01b927 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:52:30.871188 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1499b6-a9f8-442c-82f5-cf4426720ee5 None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:52:30.878735 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:52:30.937580 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cfd3883-980d-4082-8175-c66c5750d91f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:52:30.938814 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfd3883-980d-4082-8175-c66c5750d91f None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:30.939093 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfd3883-980d-4082-8175-c66c5750d91f None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:30.939380 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfd3883-980d-4082-8175-c66c5750d91f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:30.945687 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde5dcd7-0c56-435e-b222-f1198d01b927 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:52:30.953740 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cfd3883-980d-4082-8175-c66c5750d91f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 10:52:30.960635 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application [None req-4cfd3883-980d-4082-8175-c66c5750d91f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 26 10:52:30.960635 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 10:52:30.960635 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 10:52:30.960635 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 10:52:30.960635 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 10:52:30.960635 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 10:52:30.960635 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 10:52:30.960635 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 10:52:30.960635 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 10:52:30.960635 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 10:52:30.960635 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 10:52:30.960635 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 10:52:30.960635 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 10:52:30.960635 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 10:52:30.960635 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 10:52:30.960635 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 10:52:30.961691 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 10:52:30.961691 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 10:52:30.961691 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 10:52:30.961691 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 10:52:30.961691 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 10:52:30.961691 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 10:52:30.961691 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 10:52:30.961691 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 10:52:30.961691 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 10:52:30.961691 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 10:52:30.961691 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 10:52:30.961691 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application with Lock( Aug 26 10:52:30.961691 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 10:52:30.961691 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._enter() Aug 26 10:52:30.961691 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 10:52:30.961691 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 10:52:30.963193 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 10:52:30.963193 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.creator() Aug 26 10:52:30.963193 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 10:52:30.963193 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application created_value = creator( Aug 26 10:52:30.963193 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 10:52:30.963193 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 10:52:30.963193 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 10:52:30.963193 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 10:52:30.963193 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 10:52:30.963193 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 10:52:30.963193 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 26 10:52:30.963193 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Aug 26 10:52:30.963193 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 372/748] 10.222.0.40 () {56 vars in 1096 bytes} [Fri Aug 26 10:52:30 2022] GET /identity/v3/roles/admin => generated 83 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 10:52:30.963193 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cfd3883-980d-4082-8175-c66c5750d91f None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:52:30.994975 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:52:30.995916 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675eacaf-bff2-47cf-8476-69b1c44b2d5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:52:30.997169 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675eacaf-bff2-47cf-8476-69b1c44b2d5a None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:30.997481 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675eacaf-bff2-47cf-8476-69b1c44b2d5a None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:30.998017 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675eacaf-bff2-47cf-8476-69b1c44b2d5a None admin] PATH_INFO: `/v3/projects/service` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:31.009657 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-675eacaf-bff2-47cf-8476-69b1c44b2d5a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:52:31.011155 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-675eacaf-bff2-47cf-8476-69b1c44b2d5a None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:52:31.026355 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application [None req-675eacaf-bff2-47cf-8476-69b1c44b2d5a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 10:52:31.026355 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 10:52:31.026355 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 10:52:31.026355 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 10:52:31.026355 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 10:52:31.026355 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 10:52:31.026355 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 10:52:31.026355 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 10:52:31.026355 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 10:52:31.026355 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 10:52:31.026355 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 10:52:31.026355 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 10:52:31.026355 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 26 10:52:31.026355 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 26 10:52:31.026355 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 26 10:52:31.027312 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 26 10:52:31.027312 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 10:52:31.027312 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 10:52:31.027312 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 10:52:31.027312 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 10:52:31.027312 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 10:52:31.027312 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 10:52:31.027312 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 10:52:31.027312 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application with Lock( Aug 26 10:52:31.027312 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 10:52:31.027312 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._enter() Aug 26 10:52:31.027312 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 10:52:31.027312 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 10:52:31.027312 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 10:52:31.027312 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.creator() Aug 26 10:52:31.027312 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 10:52:31.027312 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application created_value = creator( Aug 26 10:52:31.028197 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 26 10:52:31.028197 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 26 10:52:31.028197 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 26 10:52:31.028197 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 26 10:52:31.028197 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 26 10:52:31.028197 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 26 10:52:31.028197 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 10:52:31.028197 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Aug 26 10:52:31.029132 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 377/749] 10.222.0.40 () {56 vars in 1111 bytes} [Fri Aug 26 10:52:30 2022] GET /identity/v3/projects/service => generated 88 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 10:52:31.029823 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675eacaf-bff2-47cf-8476-69b1c44b2d5a None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:52:31.103634 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cfd3883-980d-4082-8175-c66c5750d91f None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:52:31.152815 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91706f9a-7344-42d5-b6a3-c27bdfc89ffd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:52:31.153872 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91706f9a-7344-42d5-b6a3-c27bdfc89ffd None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:31.154187 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91706f9a-7344-42d5-b6a3-c27bdfc89ffd None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:31.154456 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91706f9a-7344-42d5-b6a3-c27bdfc89ffd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:31.157789 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675eacaf-bff2-47cf-8476-69b1c44b2d5a None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:52:31.168631 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91706f9a-7344-42d5-b6a3-c27bdfc89ffd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:52:31.169847 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:52:31.171205 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91706f9a-7344-42d5-b6a3-c27bdfc89ffd None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:52:31.189634 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application [None req-91706f9a-7344-42d5-b6a3-c27bdfc89ffd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 10:52:31.189634 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 10:52:31.189634 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 10:52:31.189634 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 10:52:31.189634 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 10:52:31.189634 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 10:52:31.189634 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 10:52:31.189634 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 10:52:31.189634 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 10:52:31.189634 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 10:52:31.189634 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 10:52:31.189634 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 10:52:31.189634 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 10:52:31.189634 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 10:52:31.189634 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 10:52:31.189634 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 10:52:31.191851 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 10:52:31.191851 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 10:52:31.191851 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 10:52:31.191851 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 10:52:31.191851 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 10:52:31.191851 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 10:52:31.191851 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 10:52:31.191851 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application with Lock( Aug 26 10:52:31.191851 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 10:52:31.191851 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._enter() Aug 26 10:52:31.191851 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 10:52:31.191851 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 10:52:31.191851 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 10:52:31.191851 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.creator() Aug 26 10:52:31.191851 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 10:52:31.191851 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application created_value = creator( Aug 26 10:52:31.191851 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 10:52:31.193732 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 10:52:31.193732 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 10:52:31.193732 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Aug 26 10:52:31.195993 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 373/750] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Aug 26 10:52:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 10:52:31.197129 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91706f9a-7344-42d5-b6a3-c27bdfc89ffd None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:52:31.218847 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb3b46f-ef2a-4fec-802b-4aa6bc0196b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:52:31.221409 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb3b46f-ef2a-4fec-802b-4aa6bc0196b7 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:31.222702 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb3b46f-ef2a-4fec-802b-4aa6bc0196b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:31.223445 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb3b46f-ef2a-4fec-802b-4aa6bc0196b7 None admin] PATH_INFO: `/v3/roles` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:31.225070 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb3b46f-ef2a-4fec-802b-4aa6bc0196b7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 10:52:31.225865 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb3b46f-ef2a-4fec-802b-4aa6bc0196b7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:52:31.229865 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb3b46f-ef2a-4fec-802b-4aa6bc0196b7 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:52:31.244159 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 378/751] 10.222.0.40 () {56 vars in 1099 bytes} [Fri Aug 26 10:52:31 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 10:52:31.246226 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb3b46f-ef2a-4fec-802b-4aa6bc0196b7 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:52:31.290637 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:52:31.318753 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91706f9a-7344-42d5-b6a3-c27bdfc89ffd None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:52:31.404563 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb3b46f-ef2a-4fec-802b-4aa6bc0196b7 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:52:31.443999 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7caef640-227f-4cac-a298-45c79cecc781 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:52:31.444560 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7caef640-227f-4cac-a298-45c79cecc781 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:31.444843 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7caef640-227f-4cac-a298-45c79cecc781 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:31.445104 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7caef640-227f-4cac-a298-45c79cecc781 None admin] PATH_INFO: `/v3/domains` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:31.445757 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7caef640-227f-4cac-a298-45c79cecc781 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 10:52:31.446086 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7caef640-227f-4cac-a298-45c79cecc781 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:52:31.447546 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7caef640-227f-4cac-a298-45c79cecc781 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:52:31.461651 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 379/752] 10.222.0.40 () {56 vars in 1109 bytes} [Fri Aug 26 10:52:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 10:52:31.466699 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7caef640-227f-4cac-a298-45c79cecc781 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:52:31.503251 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:52:31.506671 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63836fc-480b-47f4-bac4-001f875fefc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:52:31.507321 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63836fc-480b-47f4-bac4-001f875fefc6 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:31.507595 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63836fc-480b-47f4-bac4-001f875fefc6 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:31.507921 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63836fc-480b-47f4-bac4-001f875fefc6 None admin] PATH_INFO: `/v3/projects` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:31.508619 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63836fc-480b-47f4-bac4-001f875fefc6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 10:52:31.508926 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63836fc-480b-47f4-bac4-001f875fefc6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:52:31.510572 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63836fc-480b-47f4-bac4-001f875fefc6 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:52:31.530173 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 374/753] 10.222.0.40 () {56 vars in 1148 bytes} [Fri Aug 26 10:52:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 334 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 10:52:31.531494 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63836fc-480b-47f4-bac4-001f875fefc6 None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:52:31.587717 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7caef640-227f-4cac-a298-45c79cecc781 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:52:31.625596 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bbf74bf-10f4-4721-84b0-e2da26f49df2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:52:31.627264 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbf74bf-10f4-4721-84b0-e2da26f49df2 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:31.627795 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbf74bf-10f4-4721-84b0-e2da26f49df2 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:31.628269 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbf74bf-10f4-4721-84b0-e2da26f49df2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:31.638897 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bbf74bf-10f4-4721-84b0-e2da26f49df2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:52:31.640790 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bbf74bf-10f4-4721-84b0-e2da26f49df2 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:52:31.656497 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application [None req-6bbf74bf-10f4-4721-84b0-e2da26f49df2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 10:52:31.656497 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 10:52:31.656497 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 10:52:31.656497 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 10:52:31.656497 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 10:52:31.656497 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 10:52:31.656497 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 10:52:31.656497 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 10:52:31.656497 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 10:52:31.656497 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 10:52:31.656497 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 10:52:31.656497 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 10:52:31.656497 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 10:52:31.656497 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 10:52:31.656497 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 10:52:31.656497 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 10:52:31.657703 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 10:52:31.657703 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 10:52:31.657703 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 10:52:31.657703 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 10:52:31.657703 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 10:52:31.657703 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 10:52:31.657703 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 10:52:31.657703 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application with Lock( Aug 26 10:52:31.657703 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 10:52:31.657703 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._enter() Aug 26 10:52:31.657703 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 10:52:31.657703 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 10:52:31.657703 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 10:52:31.657703 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.creator() Aug 26 10:52:31.657703 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 10:52:31.657703 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application created_value = creator( Aug 26 10:52:31.657703 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 10:52:31.658625 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 10:52:31.658625 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 10:52:31.658625 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Aug 26 10:52:31.659674 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63836fc-480b-47f4-bac4-001f875fefc6 None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:52:31.660950 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 380/754] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Aug 26 10:52:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 10:52:31.662043 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bbf74bf-10f4-4721-84b0-e2da26f49df2 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:52:31.706472 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2182e1-fad7-4b5d-ad7b-8c7e9cb4584e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:52:31.707019 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2182e1-fad7-4b5d-ad7b-8c7e9cb4584e None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:31.707293 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2182e1-fad7-4b5d-ad7b-8c7e9cb4584e None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:31.707568 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2182e1-fad7-4b5d-ad7b-8c7e9cb4584e None admin] PATH_INFO: `/v3/projects/service` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:31.710294 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:52:31.716841 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd2182e1-fad7-4b5d-ad7b-8c7e9cb4584e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:52:31.719043 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd2182e1-fad7-4b5d-ad7b-8c7e9cb4584e None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:52:31.735238 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application [None req-bd2182e1-fad7-4b5d-ad7b-8c7e9cb4584e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 10:52:31.735238 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 10:52:31.735238 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 10:52:31.735238 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 10:52:31.735238 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 10:52:31.735238 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 10:52:31.735238 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 10:52:31.735238 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 10:52:31.735238 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 10:52:31.735238 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 10:52:31.735238 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 10:52:31.735238 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 10:52:31.735238 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 26 10:52:31.735238 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 26 10:52:31.735238 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 26 10:52:31.736299 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 26 10:52:31.736299 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 10:52:31.736299 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 10:52:31.736299 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 10:52:31.736299 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 10:52:31.736299 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 10:52:31.736299 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 10:52:31.736299 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 10:52:31.736299 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application with Lock( Aug 26 10:52:31.736299 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 10:52:31.736299 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._enter() Aug 26 10:52:31.736299 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 10:52:31.736299 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 10:52:31.736299 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 10:52:31.736299 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.creator() Aug 26 10:52:31.736299 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 10:52:31.736299 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application created_value = creator( Aug 26 10:52:31.737191 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 26 10:52:31.737191 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 26 10:52:31.737191 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 26 10:52:31.737191 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 26 10:52:31.737191 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 26 10:52:31.737191 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 26 10:52:31.737191 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 10:52:31.737191 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Aug 26 10:52:31.742020 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 375/755] 10.222.0.40 () {56 vars in 1111 bytes} [Fri Aug 26 10:52:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 10:52:31.742860 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2182e1-fad7-4b5d-ad7b-8c7e9cb4584e None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:52:31.768171 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:52:31.787639 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bbf74bf-10f4-4721-84b0-e2da26f49df2 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:52:31.836651 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82042a4b-155d-45cc-9587-152aef06cb47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:52:31.837256 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82042a4b-155d-45cc-9587-152aef06cb47 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:31.837828 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82042a4b-155d-45cc-9587-152aef06cb47 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:31.838138 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82042a4b-155d-45cc-9587-152aef06cb47 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:31.839143 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82042a4b-155d-45cc-9587-152aef06cb47 None admin] RBAC: Adding query filter params (role.id=1f97e4602e2e4d6498913a81f9ae8d6a, scope.project.id=0e30ccf0c1204727b786876c141dbcd2, user.id=6529bea296c9482a973ced5fe9f2212f) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 10:52:31.839449 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82042a4b-155d-45cc-9587-152aef06cb47 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:52:31.841069 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82042a4b-155d-45cc-9587-152aef06cb47 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:52:31.850136 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 381/756] 10.222.0.40 () {56 vars in 1374 bytes} [Fri Aug 26 10:52:31 2022] GET /identity/v3/role_assignments?user.id=6529bea296c9482a973ced5fe9f2212f&scope.project.id=0e30ccf0c1204727b786876c141dbcd2&role.id=1f97e4602e2e4d6498913a81f9ae8d6a => generated 259 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 10:52:31.851333 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82042a4b-155d-45cc-9587-152aef06cb47 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:52:31.888493 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2182e1-fad7-4b5d-ad7b-8c7e9cb4584e None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:52:31.978059 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede1b66b-7d14-403d-91ca-611108f24c44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:52:31.978691 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede1b66b-7d14-403d-91ca-611108f24c44 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:31.978923 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede1b66b-7d14-403d-91ca-611108f24c44 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:31.979131 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede1b66b-7d14-403d-91ca-611108f24c44 None admin] PATH_INFO: `/v3/domains` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:31.979798 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede1b66b-7d14-403d-91ca-611108f24c44 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 10:52:31.980099 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede1b66b-7d14-403d-91ca-611108f24c44 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:52:31.981491 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede1b66b-7d14-403d-91ca-611108f24c44 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:52:32.012984 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 376/757] 10.222.0.40 () {56 vars in 1109 bytes} [Fri Aug 26 10:52:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 10:52:32.024473 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede1b66b-7d14-403d-91ca-611108f24c44 None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:52:32.050544 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82042a4b-155d-45cc-9587-152aef06cb47 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:52:32.118811 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed9a9a6-9eb3-4810-b622-71b982cd2f2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:52:32.119428 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed9a9a6-9eb3-4810-b622-71b982cd2f2c None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:32.119703 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed9a9a6-9eb3-4810-b622-71b982cd2f2c None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:32.119973 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed9a9a6-9eb3-4810-b622-71b982cd2f2c None admin] PATH_INFO: `/v3/projects` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:32.120665 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed9a9a6-9eb3-4810-b622-71b982cd2f2c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 10:52:32.120969 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed9a9a6-9eb3-4810-b622-71b982cd2f2c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:52:32.122431 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed9a9a6-9eb3-4810-b622-71b982cd2f2c None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:52:32.137797 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:52:32.149355 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 382/758] 10.222.0.40 () {56 vars in 1148 bytes} [Fri Aug 26 10:52:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 298 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 10:52:32.156244 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed9a9a6-9eb3-4810-b622-71b982cd2f2c None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:52:32.179883 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:52:32.191838 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede1b66b-7d14-403d-91ca-611108f24c44 None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:52:32.244194 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b09221-0ad7-4565-b783-6a2c7ef0804d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:52:32.244751 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b09221-0ad7-4565-b783-6a2c7ef0804d None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:32.245023 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b09221-0ad7-4565-b783-6a2c7ef0804d None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:32.245284 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b09221-0ad7-4565-b783-6a2c7ef0804d None admin] PATH_INFO: `/v3/users/nova` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:32.261380 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b09221-0ad7-4565-b783-6a2c7ef0804d None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:52:32.262835 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b09221-0ad7-4565-b783-6a2c7ef0804d None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:52:32.288113 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application [None req-c4b09221-0ad7-4565-b783-6a2c7ef0804d None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 26 10:52:32.288113 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 10:52:32.288113 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 10:52:32.288113 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 10:52:32.288113 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 10:52:32.288113 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 10:52:32.288113 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 10:52:32.288113 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 10:52:32.288113 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 10:52:32.288113 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 10:52:32.288113 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 10:52:32.288113 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 10:52:32.288113 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 26 10:52:32.288113 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 26 10:52:32.288113 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 26 10:52:32.288113 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 26 10:52:32.289099 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 10:52:32.289099 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 10:52:32.289099 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 26 10:52:32.289099 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 10:52:32.289099 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 26 10:52:32.289099 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 10:52:32.289099 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 10:52:32.289099 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 10:52:32.289099 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 10:52:32.289099 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 10:52:32.289099 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 10:52:32.289099 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application with Lock( Aug 26 10:52:32.289099 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 10:52:32.289099 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._enter() Aug 26 10:52:32.289099 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 10:52:32.289099 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 10:52:32.289099 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 10:52:32.289099 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.creator() Aug 26 10:52:32.290069 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 10:52:32.290069 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application created_value = creator( Aug 26 10:52:32.290069 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 26 10:52:32.290069 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 26 10:52:32.290069 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 26 10:52:32.290069 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 26 10:52:32.290069 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 26 10:52:32.290069 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 26 10:52:32.290069 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 26 10:52:32.290069 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Aug 26 10:52:32.299920 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 377/759] 10.222.0.40 () {56 vars in 1093 bytes} [Fri Aug 26 10:52:32 2022] GET /identity/v3/users/nova => generated 82 bytes in 276 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 26 10:52:32.305083 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b09221-0ad7-4565-b783-6a2c7ef0804d None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:52:32.325090 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed9a9a6-9eb3-4810-b622-71b982cd2f2c None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:52:32.364299 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a286a24d-58fc-40dc-a875-2d75eabef9fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:52:32.364907 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a286a24d-58fc-40dc-a875-2d75eabef9fb None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:32.365180 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a286a24d-58fc-40dc-a875-2d75eabef9fb None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:32.365447 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a286a24d-58fc-40dc-a875-2d75eabef9fb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:32.366194 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a286a24d-58fc-40dc-a875-2d75eabef9fb None admin] RBAC: Adding query filter params (role.id=3bda770fcd5c46cdbfabcd7404a25e66, scope.project.id=0e30ccf0c1204727b786876c141dbcd2, user.id=50be12ccbd274a8792615e9dd10b61ca) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 10:52:32.366495 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a286a24d-58fc-40dc-a875-2d75eabef9fb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:52:32.368081 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a286a24d-58fc-40dc-a875-2d75eabef9fb None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:52:32.376812 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 383/760] 10.222.0.40 () {56 vars in 1374 bytes} [Fri Aug 26 10:52:32 2022] GET /identity/v3/role_assignments?user.id=50be12ccbd274a8792615e9dd10b61ca&scope.project.id=0e30ccf0c1204727b786876c141dbcd2&role.id=3bda770fcd5c46cdbfabcd7404a25e66 => generated 259 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 10:52:32.480773 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b09221-0ad7-4565-b783-6a2c7ef0804d None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:52:32.505112 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:52:32.538145 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d7ab63-75bd-46cd-a509-b73dd325b3fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:52:32.539111 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d7ab63-75bd-46cd-a509-b73dd325b3fb None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:32.539608 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d7ab63-75bd-46cd-a509-b73dd325b3fb None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:32.540077 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d7ab63-75bd-46cd-a509-b73dd325b3fb None admin] PATH_INFO: `/v3/users` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:32.541249 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d7ab63-75bd-46cd-a509-b73dd325b3fb None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 10:52:32.541764 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d7ab63-75bd-46cd-a509-b73dd325b3fb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:52:32.543424 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d7ab63-75bd-46cd-a509-b73dd325b3fb None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:52:32.555809 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2ba36e-2ffe-430d-956c-cbac9aac4492 None None] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:32.556316 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2ba36e-2ffe-430d-956c-cbac9aac4492 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:32.556316 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2ba36e-2ffe-430d-956c-cbac9aac4492 None None] PATH_INFO: `/` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:32.557952 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 384/761] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 26 10:52:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 26 10:52:32.565474 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 378/762] 10.222.0.40 () {56 vars in 1133 bytes} [Fri Aug 26 10:52:32 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 10:52:32.567393 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d580df-d59f-4d4a-9807-4937001c1509 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:32.567673 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d580df-d59f-4d4a-9807-4937001c1509 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:32.567940 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d580df-d59f-4d4a-9807-4937001c1509 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:32.571996 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d7ab63-75bd-46cd-a509-b73dd325b3fb None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:52:32.595747 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-47d580df-d59f-4d4a-9807-4937001c1509 None None] MFA Rules not processed for user `ca155a48e2024c32af3fd2e19e5fac11`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 10:52:32.596953 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-47d580df-d59f-4d4a-9807-4937001c1509 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 10:52:32.725433 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d7ab63-75bd-46cd-a509-b73dd325b3fb None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:52:32.728326 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 385/763] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 26 10:52:32 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 26 10:52:32.783535 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6d8f6f-df36-4159-971d-18f52a987574 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:52:32.784085 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6d8f6f-df36-4159-971d-18f52a987574 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:32.784357 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6d8f6f-df36-4159-971d-18f52a987574 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:32.784620 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6d8f6f-df36-4159-971d-18f52a987574 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:32.801308 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f6d8f6f-df36-4159-971d-18f52a987574 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:52:32.801620 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f6d8f6f-df36-4159-971d-18f52a987574 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:52:32.825822 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application [None req-4f6d8f6f-df36-4159-971d-18f52a987574 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 10:52:32.825822 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 10:52:32.825822 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 10:52:32.825822 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 10:52:32.825822 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 10:52:32.825822 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 10:52:32.825822 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 10:52:32.825822 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 10:52:32.825822 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 10:52:32.825822 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 10:52:32.825822 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 10:52:32.825822 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 10:52:32.825822 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 10:52:32.825822 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 10:52:32.825822 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 10:52:32.825822 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 10:52:32.826864 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 10:52:32.826864 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 10:52:32.826864 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 10:52:32.826864 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 10:52:32.826864 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 10:52:32.826864 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 10:52:32.826864 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 10:52:32.826864 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application with Lock( Aug 26 10:52:32.826864 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 10:52:32.826864 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._enter() Aug 26 10:52:32.826864 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 10:52:32.826864 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 10:52:32.826864 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 10:52:32.826864 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.creator() Aug 26 10:52:32.826864 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 10:52:32.826864 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application created_value = creator( Aug 26 10:52:32.826864 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 10:52:32.828185 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 10:52:32.828185 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 10:52:32.828185 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Aug 26 10:52:32.828185 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 379/764] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Aug 26 10:52:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 10:52:32.835014 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d580df-d59f-4d4a-9807-4937001c1509 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:52:32.837558 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:52:32.966744 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d580df-d59f-4d4a-9807-4937001c1509 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:52:33.005484 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e3d860-5164-49dd-bd63-ee76869e3a12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:52:33.006545 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e3d860-5164-49dd-bd63-ee76869e3a12 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:33.007030 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e3d860-5164-49dd-bd63-ee76869e3a12 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:33.007520 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e3d860-5164-49dd-bd63-ee76869e3a12 None admin] PATH_INFO: `/v3/domains` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:33.008385 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e3d860-5164-49dd-bd63-ee76869e3a12 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 10:52:33.008890 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e3d860-5164-49dd-bd63-ee76869e3a12 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:52:33.010511 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e3d860-5164-49dd-bd63-ee76869e3a12 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:52:33.028506 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 386/765] 10.222.0.40 () {56 vars in 1109 bytes} [Fri Aug 26 10:52:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 10:52:33.030237 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6d8f6f-df36-4159-971d-18f52a987574 None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:52:33.074644 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:52:33.078459 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e3d860-5164-49dd-bd63-ee76869e3a12 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:52:33.173411 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6d8f6f-df36-4159-971d-18f52a987574 None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:52:33.180457 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:52:33.221568 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16d7aff-c19d-471e-ac9e-b8cf6133290b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:52:33.223973 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e3d860-5164-49dd-bd63-ee76869e3a12 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:52:33.224808 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16d7aff-c19d-471e-ac9e-b8cf6133290b None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:33.225277 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16d7aff-c19d-471e-ac9e-b8cf6133290b None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:33.225759 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16d7aff-c19d-471e-ac9e-b8cf6133290b None admin] PATH_INFO: `/v3/projects/service` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:33.233528 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c16d7aff-c19d-471e-ac9e-b8cf6133290b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:52:33.235347 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c16d7aff-c19d-471e-ac9e-b8cf6133290b None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:52:33.247656 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application [None req-c16d7aff-c19d-471e-ac9e-b8cf6133290b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 10:52:33.247656 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 10:52:33.247656 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 10:52:33.247656 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 10:52:33.247656 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 10:52:33.247656 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 10:52:33.247656 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 10:52:33.247656 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 10:52:33.247656 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 10:52:33.247656 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 10:52:33.247656 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 10:52:33.247656 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 10:52:33.247656 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 26 10:52:33.247656 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 26 10:52:33.247656 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 26 10:52:33.248596 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 26 10:52:33.248596 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 10:52:33.248596 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 10:52:33.248596 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 10:52:33.248596 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 10:52:33.248596 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 10:52:33.248596 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 10:52:33.248596 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 10:52:33.248596 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application with Lock( Aug 26 10:52:33.248596 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 10:52:33.248596 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._enter() Aug 26 10:52:33.248596 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 10:52:33.248596 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 10:52:33.248596 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 10:52:33.248596 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.creator() Aug 26 10:52:33.248596 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 10:52:33.248596 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application created_value = creator( Aug 26 10:52:33.249621 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 26 10:52:33.249621 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 26 10:52:33.249621 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 26 10:52:33.249621 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 26 10:52:33.249621 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 26 10:52:33.249621 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 26 10:52:33.249621 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 10:52:33.249621 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Aug 26 10:52:33.252898 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 380/766] 10.222.0.40 () {56 vars in 1111 bytes} [Fri Aug 26 10:52:33 2022] GET /identity/v3/projects/service => generated 88 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 10:52:33.254321 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd337f9c-471e-4073-8f22-430480e9bbf0 None None] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:33.254810 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd337f9c-471e-4073-8f22-430480e9bbf0 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:33.255275 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd337f9c-471e-4073-8f22-430480e9bbf0 None None] PATH_INFO: `/` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:33.256928 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 381/767] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 26 10:52:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 10:52:33.265272 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4d2432-daba-40b0-bb33-ec8e4af44197 None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:33.265272 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4d2432-daba-40b0-bb33-ec8e4af44197 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:33.265272 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4d2432-daba-40b0-bb33-ec8e4af44197 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:33.270186 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2903f871-a529-4196-8ad7-f0700606cfe3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:52:33.271083 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2903f871-a529-4196-8ad7-f0700606cfe3 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:33.271544 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2903f871-a529-4196-8ad7-f0700606cfe3 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:33.271996 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2903f871-a529-4196-8ad7-f0700606cfe3 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:33.272779 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2903f871-a529-4196-8ad7-f0700606cfe3 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:52:33.274435 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2903f871-a529-4196-8ad7-f0700606cfe3 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:52:33.283301 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-ff4d2432-daba-40b0-bb33-ec8e4af44197 None None] MFA Rules not processed for user `ca155a48e2024c32af3fd2e19e5fac11`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 10:52:33.284489 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-ff4d2432-daba-40b0-bb33-ec8e4af44197 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 10:52:33.287336 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application [None req-2903f871-a529-4196-8ad7-f0700606cfe3 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 26 10:52:33.287336 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 10:52:33.287336 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 26 10:52:33.287336 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 26 10:52:33.287336 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 26 10:52:33.287336 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 26 10:52:33.287336 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 26 10:52:33.287336 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 26 10:52:33.287336 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 26 10:52:33.287336 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Aug 26 10:52:33.287336 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 26 10:52:33.287336 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Aug 26 10:52:33.287336 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 10:52:33.287336 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 10:52:33.287336 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 10:52:33.287336 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 10:52:33.287336 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 10:52:33.288333 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 10:52:33.288333 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 10:52:33.288333 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 10:52:33.288333 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 10:52:33.288333 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 10:52:33.288333 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 10:52:33.288333 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 26 10:52:33.288333 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 26 10:52:33.288333 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 26 10:52:33.288333 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 26 10:52:33.288333 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 10:52:33.288333 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 10:52:33.288333 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 10:52:33.288333 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 10:52:33.288333 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 10:52:33.288333 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 10:52:33.289203 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 10:52:33.289203 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application with Lock( Aug 26 10:52:33.289203 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 10:52:33.289203 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._enter() Aug 26 10:52:33.289203 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 10:52:33.289203 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 10:52:33.289203 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 10:52:33.289203 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.creator() Aug 26 10:52:33.289203 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 10:52:33.289203 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application created_value = creator( Aug 26 10:52:33.289203 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 26 10:52:33.289203 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 26 10:52:33.289203 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 26 10:52:33.289203 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Aug 26 10:52:33.292035 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 387/768] 10.222.0.40 () {56 vars in 1129 bytes} [Fri Aug 26 10:52:33 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 10:52:33.293062 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2903f871-a529-4196-8ad7-f0700606cfe3 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:52:33.369579 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 382/769] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 26 10:52:33 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 26 10:52:33.370600 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4d2432-daba-40b0-bb33-ec8e4af44197 None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:52:33.398929 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:52:33.400659 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2903f871-a529-4196-8ad7-f0700606cfe3 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:52:33.431833 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:52:33.449898 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122b7809-b419-452c-97ad-c899d1a29842 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:52:33.450826 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122b7809-b419-452c-97ad-c899d1a29842 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:33.451321 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122b7809-b419-452c-97ad-c899d1a29842 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:33.451774 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122b7809-b419-452c-97ad-c899d1a29842 None admin] PATH_INFO: `/v3/projects` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:33.452645 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-122b7809-b419-452c-97ad-c899d1a29842 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 10:52:33.453578 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-122b7809-b419-452c-97ad-c899d1a29842 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:52:33.455797 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-122b7809-b419-452c-97ad-c899d1a29842 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:52:33.486115 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 388/770] 10.222.0.40 () {56 vars in 1148 bytes} [Fri Aug 26 10:52:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 10:52:33.488835 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122b7809-b419-452c-97ad-c899d1a29842 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:52:33.584771 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4d2432-daba-40b0-bb33-ec8e4af44197 None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:52:33.614430 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122b7809-b419-452c-97ad-c899d1a29842 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:52:33.621093 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7541682-28a1-4a77-8690-772e874fbc59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:52:33.621759 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7541682-28a1-4a77-8690-772e874fbc59 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:33.622122 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7541682-28a1-4a77-8690-772e874fbc59 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:33.622458 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7541682-28a1-4a77-8690-772e874fbc59 None admin] PATH_INFO: `/v3/services` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:33.623199 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7541682-28a1-4a77-8690-772e874fbc59 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 10:52:33.623936 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7541682-28a1-4a77-8690-772e874fbc59 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:52:33.625086 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7541682-28a1-4a77-8690-772e874fbc59 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:52:33.631206 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 383/771] 10.222.0.40 () {56 vars in 1124 bytes} [Fri Aug 26 10:52:33 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 10:52:33.641669 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7541682-28a1-4a77-8690-772e874fbc59 None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:52:33.656390 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfaf8710-d013-4c34-a4eb-536c1a88ebd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:52:33.657340 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfaf8710-d013-4c34-a4eb-536c1a88ebd0 None admin] REQUEST_METHOD: `PUT` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:33.657996 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfaf8710-d013-4c34-a4eb-536c1a88ebd0 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:33.658422 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfaf8710-d013-4c34-a4eb-536c1a88ebd0 None admin] PATH_INFO: `/v3/projects/0e30ccf0c1204727b786876c141dbcd2/users/1f4b4a662db049ecacc865ecff10bdb0/roles/1f97e4602e2e4d6498913a81f9ae8d6a` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:33.660655 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfaf8710-d013-4c34-a4eb-536c1a88ebd0 None admin] RBAC: Authorizing `identity:create_grant(project_id=0e30ccf0c1204727b786876c141dbcd2, user_id=1f4b4a662db049ecacc865ecff10bdb0, role_id=1f97e4602e2e4d6498913a81f9ae8d6a)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:52:33.662424 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfaf8710-d013-4c34-a4eb-536c1a88ebd0 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:52:33.682522 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 389/772] 10.222.0.40 () {58 vars in 1439 bytes} [Fri Aug 26 10:52:33 2022] PUT /identity/v3/projects/0e30ccf0c1204727b786876c141dbcd2/users/1f4b4a662db049ecacc865ecff10bdb0/roles/1f97e4602e2e4d6498913a81f9ae8d6a => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 26 10:52:33.731934 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfaf8710-d013-4c34-a4eb-536c1a88ebd0 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:52:33.813782 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:52:33.833159 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:52:33.848577 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7541682-28a1-4a77-8690-772e874fbc59 None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:52:33.888987 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0cc272-264d-4485-ae32-708722c8d5f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:52:33.889840 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0cc272-264d-4485-ae32-708722c8d5f4 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:33.890166 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0cc272-264d-4485-ae32-708722c8d5f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:33.890443 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0cc272-264d-4485-ae32-708722c8d5f4 None admin] PATH_INFO: `/v3/services` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:33.891087 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c0cc272-264d-4485-ae32-708722c8d5f4 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 10:52:33.891963 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c0cc272-264d-4485-ae32-708722c8d5f4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:52:33.893457 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c0cc272-264d-4485-ae32-708722c8d5f4 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:52:33.899957 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 384/773] 10.222.0.40 () {56 vars in 1124 bytes} [Fri Aug 26 10:52:33 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 10:52:33.906069 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0cc272-264d-4485-ae32-708722c8d5f4 None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:52:33.951098 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfaf8710-d013-4c34-a4eb-536c1a88ebd0 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:52:34.005802 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2018d16-0e4a-419d-b985-b2938540d66f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:52:34.006889 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2018d16-0e4a-419d-b985-b2938540d66f None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:34.007363 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2018d16-0e4a-419d-b985-b2938540d66f None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:34.007827 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2018d16-0e4a-419d-b985-b2938540d66f None admin] PATH_INFO: `/v3/services/network` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:34.008606 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2018d16-0e4a-419d-b985-b2938540d66f None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:52:34.010219 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2018d16-0e4a-419d-b985-b2938540d66f None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:52:34.020902 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application [None req-d2018d16-0e4a-419d-b985-b2938540d66f None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 26 10:52:34.020902 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 10:52:34.020902 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 26 10:52:34.020902 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 26 10:52:34.020902 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 26 10:52:34.020902 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 26 10:52:34.020902 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 26 10:52:34.020902 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 26 10:52:34.020902 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 26 10:52:34.020902 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Aug 26 10:52:34.020902 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 26 10:52:34.020902 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Aug 26 10:52:34.020902 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 10:52:34.020902 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 10:52:34.020902 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 10:52:34.020902 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 10:52:34.020902 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 10:52:34.021916 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 10:52:34.021916 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 10:52:34.021916 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 10:52:34.021916 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 10:52:34.021916 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 10:52:34.021916 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 10:52:34.021916 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 26 10:52:34.021916 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 26 10:52:34.021916 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 26 10:52:34.021916 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 26 10:52:34.021916 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 10:52:34.021916 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 10:52:34.021916 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 10:52:34.021916 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 10:52:34.021916 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 10:52:34.021916 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 10:52:34.022778 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 10:52:34.022778 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application with Lock( Aug 26 10:52:34.022778 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 10:52:34.022778 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._enter() Aug 26 10:52:34.022778 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 10:52:34.022778 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 10:52:34.022778 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 10:52:34.022778 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.creator() Aug 26 10:52:34.022778 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 10:52:34.022778 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application created_value = creator( Aug 26 10:52:34.022778 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 26 10:52:34.022778 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 26 10:52:34.022778 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 26 10:52:34.022778 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Aug 26 10:52:34.025534 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 390/774] 10.222.0.40 () {56 vars in 1111 bytes} [Fri Aug 26 10:52:33 2022] GET /identity/v3/services/network => generated 88 bytes in 294 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 26 10:52:34.034029 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2018d16-0e4a-419d-b985-b2938540d66f None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:52:34.116516 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0cc272-264d-4485-ae32-708722c8d5f4 None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:52:34.169323 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6cfd8a-2d54-4a54-bfd4-d8c814dd51cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:52:34.170297 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6cfd8a-2d54-4a54-bfd4-d8c814dd51cb None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:34.171556 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6cfd8a-2d54-4a54-bfd4-d8c814dd51cb None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:34.173547 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6cfd8a-2d54-4a54-bfd4-d8c814dd51cb None admin] PATH_INFO: `/v3/endpoints` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:34.174679 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f6cfd8a-2d54-4a54-bfd4-d8c814dd51cb None admin] RBAC: Adding query filter params (interface=public, service_id=f0be199e96104c0f83b457e0aa2596e9, region_id=RegionOne) {{(pid=81423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 10:52:34.177119 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2018d16-0e4a-419d-b985-b2938540d66f None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:52:34.177643 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f6cfd8a-2d54-4a54-bfd4-d8c814dd51cb None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:52:34.179234 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f6cfd8a-2d54-4a54-bfd4-d8c814dd51cb None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:52:34.188583 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 385/775] 10.222.0.40 () {56 vars in 1251 bytes} [Fri Aug 26 10:52:33 2022] GET /identity/v3/endpoints?service_id=f0be199e96104c0f83b457e0aa2596e9&interface=public®ion_id=RegionOne => generated 194 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 10:52:34.189927 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231ce1d2-ff93-4a74-9399-8305b54c7a12 None None] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:34.194433 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231ce1d2-ff93-4a74-9399-8305b54c7a12 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:34.194848 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231ce1d2-ff93-4a74-9399-8305b54c7a12 None None] PATH_INFO: `/` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:34.195213 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:52:34.200323 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 386/776] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 26 10:52:34 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 10:52:34.200323 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231ce1d2-ff93-4a74-9399-8305b54c7a12 None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:52:34.211945 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:52:34.233842 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49de3cb2-f750-4f51-a6dc-71bfd89c8b28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:52:34.234957 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49de3cb2-f750-4f51-a6dc-71bfd89c8b28 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:34.235497 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49de3cb2-f750-4f51-a6dc-71bfd89c8b28 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:34.236313 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49de3cb2-f750-4f51-a6dc-71bfd89c8b28 None admin] PATH_INFO: `/v3/services` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:34.237582 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49de3cb2-f750-4f51-a6dc-71bfd89c8b28 None admin] RBAC: Adding query filter params (name=network) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 10:52:34.238177 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49de3cb2-f750-4f51-a6dc-71bfd89c8b28 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:52:34.240135 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49de3cb2-f750-4f51-a6dc-71bfd89c8b28 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:52:34.246796 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 391/777] 10.222.0.40 () {56 vars in 1112 bytes} [Fri Aug 26 10:52:34 2022] GET /identity/v3/services?name=network => generated 124 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 10:52:34.248182 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b76529c-c846-49f8-ae7b-7e888eba3f3e None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:34.248589 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b76529c-c846-49f8-ae7b-7e888eba3f3e None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:34.248994 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b76529c-c846-49f8-ae7b-7e888eba3f3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:34.275120 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-7b76529c-c846-49f8-ae7b-7e888eba3f3e None None] MFA Rules not processed for user `ca155a48e2024c32af3fd2e19e5fac11`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 10:52:34.276607 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-7b76529c-c846-49f8-ae7b-7e888eba3f3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 10:52:34.334198 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231ce1d2-ff93-4a74-9399-8305b54c7a12 None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:52:34.361999 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 392/778] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 26 10:52:34 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 26 10:52:34.361999 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b76529c-c846-49f8-ae7b-7e888eba3f3e None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:52:34.388085 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1afaa9ba-fa39-4d3f-a541-82e44ace86b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:52:34.388996 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afaa9ba-fa39-4d3f-a541-82e44ace86b0 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:34.389471 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afaa9ba-fa39-4d3f-a541-82e44ace86b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:34.389911 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afaa9ba-fa39-4d3f-a541-82e44ace86b0 None admin] PATH_INFO: `/v3/services` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:34.390730 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1afaa9ba-fa39-4d3f-a541-82e44ace86b0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:52:34.393259 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1afaa9ba-fa39-4d3f-a541-82e44ace86b0 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:52:34.408493 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 387/779] 10.222.0.40 () {56 vars in 1087 bytes} [Fri Aug 26 10:52:34 2022] GET /identity/v3/services => generated 772 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 10:52:34.469154 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b76529c-c846-49f8-ae7b-7e888eba3f3e None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:52:34.472261 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:52:34.508313 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ccb8dd-83b6-46ba-931e-27e737acf924 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:52:34.508933 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ccb8dd-83b6-46ba-931e-27e737acf924 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:34.509211 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ccb8dd-83b6-46ba-931e-27e737acf924 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:34.509476 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ccb8dd-83b6-46ba-931e-27e737acf924 None admin] PATH_INFO: `/v3/services` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:34.510146 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ccb8dd-83b6-46ba-931e-27e737acf924 None admin] RBAC: Adding query filter params (type=network) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 10:52:34.510450 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ccb8dd-83b6-46ba-931e-27e737acf924 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:52:34.511821 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ccb8dd-83b6-46ba-931e-27e737acf924 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:52:34.519049 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 393/780] 10.222.0.40 () {56 vars in 1112 bytes} [Fri Aug 26 10:52:34 2022] GET /identity/v3/services?type=network => generated 350 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 10:52:34.525132 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1afaa9ba-fa39-4d3f-a541-82e44ace86b0 None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:52:34.629473 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:52:34.653042 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace32543-975d-469f-a298-73a33e7fcc06 None None] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:34.653553 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace32543-975d-469f-a298-73a33e7fcc06 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:34.653627 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace32543-975d-469f-a298-73a33e7fcc06 None None] PATH_INFO: `/` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:34.656331 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 394/781] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 26 10:52:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 10:52:34.660755 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410d8882-7ae8-4de6-bef2-0a3d779644af None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:34.661039 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410d8882-7ae8-4de6-bef2-0a3d779644af None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:34.661392 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410d8882-7ae8-4de6-bef2-0a3d779644af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:34.673167 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1afaa9ba-fa39-4d3f-a541-82e44ace86b0 None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:52:34.693822 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-410d8882-7ae8-4de6-bef2-0a3d779644af None None] MFA Rules not processed for user `ca155a48e2024c32af3fd2e19e5fac11`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 10:52:34.695159 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-410d8882-7ae8-4de6-bef2-0a3d779644af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 10:52:34.722097 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e535be-fd35-4887-a6d0-b941f68af9ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:52:34.722411 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e535be-fd35-4887-a6d0-b941f68af9ac None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:34.722687 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e535be-fd35-4887-a6d0-b941f68af9ac None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:34.722954 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e535be-fd35-4887-a6d0-b941f68af9ac None admin] PATH_INFO: `/v3/endpoints` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:34.724051 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e535be-fd35-4887-a6d0-b941f68af9ac None admin] RBAC: Adding query filter params (interface=public, service_id=ea246a4f02b2490fb592a75b748f1929, region_id=RegionOne) {{(pid=81423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 10:52:34.724619 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e535be-fd35-4887-a6d0-b941f68af9ac None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:52:34.726695 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e535be-fd35-4887-a6d0-b941f68af9ac None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:52:34.734106 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 388/782] 10.222.0.40 () {56 vars in 1251 bytes} [Fri Aug 26 10:52:34 2022] GET /identity/v3/endpoints?service_id=ea246a4f02b2490fb592a75b748f1929&interface=public®ion_id=RegionOne => generated 194 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 10:52:34.734760 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e535be-fd35-4887-a6d0-b941f68af9ac None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:52:34.788230 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 395/783] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 26 10:52:34 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 26 10:52:34.789105 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410d8882-7ae8-4de6-bef2-0a3d779644af None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:52:34.912939 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e535be-fd35-4887-a6d0-b941f68af9ac None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:52:34.993406 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90500e82-ff69-4eba-a4ab-5ad817c0d70e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:52:34.995928 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90500e82-ff69-4eba-a4ab-5ad817c0d70e None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:34.996369 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90500e82-ff69-4eba-a4ab-5ad817c0d70e None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:34.997887 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90500e82-ff69-4eba-a4ab-5ad817c0d70e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:35.010394 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:52:35.023432 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90500e82-ff69-4eba-a4ab-5ad817c0d70e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 10:52:35.035629 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:52:35.039776 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410d8882-7ae8-4de6-bef2-0a3d779644af None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:52:35.042296 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application [None req-90500e82-ff69-4eba-a4ab-5ad817c0d70e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 26 10:52:35.042296 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 10:52:35.042296 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 10:52:35.042296 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 10:52:35.042296 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 10:52:35.042296 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 10:52:35.042296 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 10:52:35.042296 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 10:52:35.042296 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 10:52:35.042296 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 10:52:35.042296 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 10:52:35.042296 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 10:52:35.042296 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 10:52:35.042296 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 10:52:35.042296 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 10:52:35.042296 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 10:52:35.043227 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 10:52:35.043227 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 10:52:35.043227 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 10:52:35.043227 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 10:52:35.043227 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 10:52:35.043227 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 10:52:35.043227 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 10:52:35.043227 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 10:52:35.043227 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 10:52:35.043227 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 10:52:35.043227 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 10:52:35.043227 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application with Lock( Aug 26 10:52:35.043227 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 10:52:35.043227 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._enter() Aug 26 10:52:35.043227 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 10:52:35.043227 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 10:52:35.044467 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 10:52:35.044467 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.creator() Aug 26 10:52:35.044467 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 10:52:35.044467 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application created_value = creator( Aug 26 10:52:35.044467 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 10:52:35.044467 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 10:52:35.044467 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 10:52:35.044467 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 10:52:35.044467 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 10:52:35.044467 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 10:52:35.044467 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 26 10:52:35.044467 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Aug 26 10:52:35.051982 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 389/784] 10.222.0.40 () {56 vars in 1096 bytes} [Fri Aug 26 10:52:34 2022] GET /identity/v3/roles/admin => generated 83 bytes in 317 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 10:52:35.054425 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90500e82-ff69-4eba-a4ab-5ad817c0d70e None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:52:35.120486 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0f892e-20f5-43a3-bbcb-118aac6d9f77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:52:35.122655 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0f892e-20f5-43a3-bbcb-118aac6d9f77 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:35.124091 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0f892e-20f5-43a3-bbcb-118aac6d9f77 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:35.124600 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0f892e-20f5-43a3-bbcb-118aac6d9f77 None admin] PATH_INFO: `/v3/services` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:35.125413 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce0f892e-20f5-43a3-bbcb-118aac6d9f77 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:52:35.128040 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce0f892e-20f5-43a3-bbcb-118aac6d9f77 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:52:35.144597 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 396/785] 10.222.0.40 () {56 vars in 1087 bytes} [Fri Aug 26 10:52:34 2022] GET /identity/v3/services => generated 772 bytes in 356 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 10:52:35.182270 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0f892e-20f5-43a3-bbcb-118aac6d9f77 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:52:35.258680 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90500e82-ff69-4eba-a4ab-5ad817c0d70e None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:52:35.368431 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd48acd1-1834-4ce1-b1bf-4695ee35c3de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:52:35.369599 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd48acd1-1834-4ce1-b1bf-4695ee35c3de None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:35.370115 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd48acd1-1834-4ce1-b1bf-4695ee35c3de None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:35.370588 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd48acd1-1834-4ce1-b1bf-4695ee35c3de None admin] PATH_INFO: `/v3/roles` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:35.372019 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd48acd1-1834-4ce1-b1bf-4695ee35c3de None admin] RBAC: Adding query filter params (name=admin) {{(pid=81423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 10:52:35.372528 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd48acd1-1834-4ce1-b1bf-4695ee35c3de None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:52:35.374406 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd48acd1-1834-4ce1-b1bf-4695ee35c3de None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:52:35.389783 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 390/786] 10.222.0.40 () {56 vars in 1099 bytes} [Fri Aug 26 10:52:35 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 336 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 10:52:35.397559 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd48acd1-1834-4ce1-b1bf-4695ee35c3de None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:52:35.432562 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0f892e-20f5-43a3-bbcb-118aac6d9f77 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:52:35.475434 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1633ea8-9233-4935-8016-4e98c0d6c665 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:52:35.476469 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1633ea8-9233-4935-8016-4e98c0d6c665 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:35.476950 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1633ea8-9233-4935-8016-4e98c0d6c665 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:35.477424 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1633ea8-9233-4935-8016-4e98c0d6c665 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:35.489574 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1633ea8-9233-4935-8016-4e98c0d6c665 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 10:52:35.495287 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application [None req-e1633ea8-9233-4935-8016-4e98c0d6c665 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 26 10:52:35.495287 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 10:52:35.495287 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 10:52:35.495287 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 10:52:35.495287 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 10:52:35.495287 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 10:52:35.495287 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 10:52:35.495287 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 10:52:35.495287 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 10:52:35.495287 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 10:52:35.495287 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 10:52:35.495287 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 10:52:35.495287 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 10:52:35.495287 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 10:52:35.495287 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 10:52:35.495287 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 10:52:35.496242 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 10:52:35.496242 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 10:52:35.496242 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 10:52:35.496242 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 10:52:35.496242 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 10:52:35.496242 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 10:52:35.496242 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 10:52:35.496242 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 10:52:35.496242 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 10:52:35.496242 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 10:52:35.496242 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 10:52:35.496242 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application with Lock( Aug 26 10:52:35.496242 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 10:52:35.496242 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._enter() Aug 26 10:52:35.496242 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 10:52:35.496242 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 10:52:35.497107 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 10:52:35.497107 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.creator() Aug 26 10:52:35.497107 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 10:52:35.497107 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application created_value = creator( Aug 26 10:52:35.497107 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 10:52:35.497107 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 10:52:35.497107 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 10:52:35.497107 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 10:52:35.497107 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 10:52:35.497107 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 10:52:35.497107 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 26 10:52:35.497107 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Aug 26 10:52:35.499910 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 397/787] 10.222.0.40 () {56 vars in 1102 bytes} [Fri Aug 26 10:52:35 2022] GET /identity/v3/roles/service => generated 85 bytes in 318 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 26 10:52:35.511449 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1633ea8-9233-4935-8016-4e98c0d6c665 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:52:35.550350 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:52:35.578323 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:52:35.656327 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd48acd1-1834-4ce1-b1bf-4695ee35c3de None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:52:35.702653 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6498ffd-a792-40bb-830d-69270a314b33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:52:35.703283 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6498ffd-a792-40bb-830d-69270a314b33 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:35.703561 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6498ffd-a792-40bb-830d-69270a314b33 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:35.703844 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6498ffd-a792-40bb-830d-69270a314b33 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:35.706027 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1633ea8-9233-4935-8016-4e98c0d6c665 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:52:35.730006 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6498ffd-a792-40bb-830d-69270a314b33 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:52:35.731472 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6498ffd-a792-40bb-830d-69270a314b33 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:52:35.759432 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application [None req-f6498ffd-a792-40bb-830d-69270a314b33 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 10:52:35.759432 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 10:52:35.759432 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 10:52:35.759432 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 10:52:35.759432 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 10:52:35.759432 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 10:52:35.759432 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 10:52:35.759432 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 10:52:35.759432 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 10:52:35.759432 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 10:52:35.759432 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 10:52:35.759432 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 10:52:35.759432 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 10:52:35.759432 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 10:52:35.759432 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 10:52:35.759432 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 10:52:35.760425 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 10:52:35.760425 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 10:52:35.760425 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 10:52:35.760425 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 10:52:35.760425 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 10:52:35.760425 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 10:52:35.760425 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 10:52:35.760425 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application with Lock( Aug 26 10:52:35.760425 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 10:52:35.760425 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._enter() Aug 26 10:52:35.760425 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 10:52:35.760425 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 10:52:35.760425 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 10:52:35.760425 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.creator() Aug 26 10:52:35.760425 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 10:52:35.760425 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application created_value = creator( Aug 26 10:52:35.760425 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 10:52:35.761296 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 10:52:35.761296 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 10:52:35.761296 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Aug 26 10:52:35.772354 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 391/788] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Aug 26 10:52:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 372 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 10:52:35.774655 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6498ffd-a792-40bb-830d-69270a314b33 None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:52:35.786812 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d7d9e7-b4b7-489c-b8b2-06f7ce0add52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:52:35.787441 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d7d9e7-b4b7-489c-b8b2-06f7ce0add52 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:35.787717 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d7d9e7-b4b7-489c-b8b2-06f7ce0add52 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:35.787987 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d7d9e7-b4b7-489c-b8b2-06f7ce0add52 None admin] PATH_INFO: `/v3/roles` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:35.788666 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09d7d9e7-b4b7-489c-b8b2-06f7ce0add52 None admin] RBAC: Adding query filter params (name=service) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 10:52:35.788966 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09d7d9e7-b4b7-489c-b8b2-06f7ce0add52 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:52:35.790378 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09d7d9e7-b4b7-489c-b8b2-06f7ce0add52 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:52:35.802444 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 398/789] 10.222.0.40 () {56 vars in 1103 bytes} [Fri Aug 26 10:52:35 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 291 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 10:52:35.806887 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d7d9e7-b4b7-489c-b8b2-06f7ce0add52 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:52:35.934086 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6498ffd-a792-40bb-830d-69270a314b33 None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:52:35.942504 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:52:35.950215 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:52:35.962880 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d7d9e7-b4b7-489c-b8b2-06f7ce0add52 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:52:35.977046 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0bfc4c-21da-44d5-8039-c07280c40bfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:52:35.977790 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0bfc4c-21da-44d5-8039-c07280c40bfd None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:35.978204 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0bfc4c-21da-44d5-8039-c07280c40bfd None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:35.978577 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0bfc4c-21da-44d5-8039-c07280c40bfd None admin] PATH_INFO: `/v3/domains` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:35.979345 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f0bfc4c-21da-44d5-8039-c07280c40bfd None admin] RBAC: Adding query filter params (name=Default) {{(pid=81423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 10:52:35.979745 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f0bfc4c-21da-44d5-8039-c07280c40bfd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:52:35.981189 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f0bfc4c-21da-44d5-8039-c07280c40bfd None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:52:35.994696 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 392/790] 10.222.0.40 () {56 vars in 1109 bytes} [Fri Aug 26 10:52:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 10:52:35.996146 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d388a37-3106-456f-9d48-79b7f912ec29 None None] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:35.996621 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d388a37-3106-456f-9d48-79b7f912ec29 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:35.997075 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d388a37-3106-456f-9d48-79b7f912ec29 None None] PATH_INFO: `/` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:35.999071 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 393/791] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 26 10:52:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 10:52:35.999923 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d388a37-3106-456f-9d48-79b7f912ec29 None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:52:36.004507 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64e1a7f-1fe9-4150-9c31-be75d9f11856 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:52:36.005169 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64e1a7f-1fe9-4150-9c31-be75d9f11856 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:36.005452 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64e1a7f-1fe9-4150-9c31-be75d9f11856 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:36.005731 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64e1a7f-1fe9-4150-9c31-be75d9f11856 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:36.017331 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c64e1a7f-1fe9-4150-9c31-be75d9f11856 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:52:36.018809 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c64e1a7f-1fe9-4150-9c31-be75d9f11856 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:52:36.034028 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application [None req-c64e1a7f-1fe9-4150-9c31-be75d9f11856 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 10:52:36.034028 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 10:52:36.034028 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 10:52:36.034028 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 10:52:36.034028 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 10:52:36.034028 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 10:52:36.034028 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 10:52:36.034028 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 10:52:36.034028 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 10:52:36.034028 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 10:52:36.034028 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 10:52:36.034028 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 10:52:36.034028 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 10:52:36.034028 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 10:52:36.034028 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 10:52:36.034028 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 10:52:36.036238 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 10:52:36.036238 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 10:52:36.036238 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 10:52:36.036238 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 10:52:36.036238 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 10:52:36.036238 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 10:52:36.036238 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 10:52:36.036238 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application with Lock( Aug 26 10:52:36.036238 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 10:52:36.036238 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._enter() Aug 26 10:52:36.036238 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 10:52:36.036238 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 10:52:36.036238 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 10:52:36.036238 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.creator() Aug 26 10:52:36.036238 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 10:52:36.036238 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application created_value = creator( Aug 26 10:52:36.036238 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 10:52:36.038482 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 10:52:36.038482 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 10:52:36.038482 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Aug 26 10:52:36.042715 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 399/792] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Aug 26 10:52:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 26 10:52:36.042715 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b841a3-769b-4c16-b42c-989ba1bebc50 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:36.042715 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b841a3-769b-4c16-b42c-989ba1bebc50 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:36.042715 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b841a3-769b-4c16-b42c-989ba1bebc50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:36.060932 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-43b841a3-769b-4c16-b42c-989ba1bebc50 None None] MFA Rules not processed for user `ca155a48e2024c32af3fd2e19e5fac11`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 10:52:36.062166 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-43b841a3-769b-4c16-b42c-989ba1bebc50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 10:52:36.111363 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d388a37-3106-456f-9d48-79b7f912ec29 None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:52:36.140726 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 400/793] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 26 10:52:36 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 26 10:52:36.142298 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b841a3-769b-4c16-b42c-989ba1bebc50 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:52:36.146624 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89a80b7-3999-415a-a735-eceb52a4e688 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:52:36.147264 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89a80b7-3999-415a-a735-eceb52a4e688 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:36.147541 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89a80b7-3999-415a-a735-eceb52a4e688 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:36.147812 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89a80b7-3999-415a-a735-eceb52a4e688 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:36.155245 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f89a80b7-3999-415a-a735-eceb52a4e688 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:52:36.156674 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f89a80b7-3999-415a-a735-eceb52a4e688 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:52:36.169325 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application [None req-f89a80b7-3999-415a-a735-eceb52a4e688 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 26 10:52:36.169325 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 10:52:36.169325 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 10:52:36.169325 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 10:52:36.169325 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 10:52:36.169325 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 10:52:36.169325 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 10:52:36.169325 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 10:52:36.169325 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 10:52:36.169325 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 10:52:36.169325 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 10:52:36.169325 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 10:52:36.169325 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 26 10:52:36.169325 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 26 10:52:36.169325 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 26 10:52:36.169325 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 26 10:52:36.171268 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 10:52:36.171268 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 10:52:36.171268 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 26 10:52:36.171268 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 10:52:36.171268 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 26 10:52:36.171268 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 10:52:36.171268 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 10:52:36.171268 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 10:52:36.171268 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 10:52:36.171268 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 10:52:36.171268 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 10:52:36.171268 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application with Lock( Aug 26 10:52:36.171268 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 10:52:36.171268 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._enter() Aug 26 10:52:36.171268 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 10:52:36.171268 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 10:52:36.171268 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 10:52:36.171268 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.creator() Aug 26 10:52:36.172493 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 10:52:36.172493 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application created_value = creator( Aug 26 10:52:36.172493 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 26 10:52:36.172493 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 26 10:52:36.172493 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 26 10:52:36.172493 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 26 10:52:36.172493 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 26 10:52:36.172493 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 26 10:52:36.172493 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 26 10:52:36.172493 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Aug 26 10:52:36.172493 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 394/794] 10.222.0.40 () {56 vars in 1096 bytes} [Fri Aug 26 10:52:35 2022] GET /identity/v3/users/swift => generated 83 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 10:52:36.177451 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89a80b7-3999-415a-a735-eceb52a4e688 None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:52:36.244057 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:52:36.267317 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:52:36.322203 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b841a3-769b-4c16-b42c-989ba1bebc50 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:52:36.364983 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992b7d82-eb09-4732-accc-11532c8bb5b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:52:36.366585 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992b7d82-eb09-4732-accc-11532c8bb5b7 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:36.367567 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992b7d82-eb09-4732-accc-11532c8bb5b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:36.367943 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992b7d82-eb09-4732-accc-11532c8bb5b7 None admin] PATH_INFO: `/v3/domains` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:36.368892 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-992b7d82-eb09-4732-accc-11532c8bb5b7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 10:52:36.369945 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-992b7d82-eb09-4732-accc-11532c8bb5b7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:52:36.371603 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-992b7d82-eb09-4732-accc-11532c8bb5b7 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:52:36.387182 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89a80b7-3999-415a-a735-eceb52a4e688 None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:52:36.397199 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 401/795] 10.222.0.40 () {56 vars in 1109 bytes} [Fri Aug 26 10:52:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 10:52:36.398204 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992b7d82-eb09-4732-accc-11532c8bb5b7 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:52:36.453177 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571718ae-f98b-43d4-82c1-e609064eb77d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:52:36.454833 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571718ae-f98b-43d4-82c1-e609064eb77d None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:36.456320 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571718ae-f98b-43d4-82c1-e609064eb77d None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:36.456740 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571718ae-f98b-43d4-82c1-e609064eb77d None admin] PATH_INFO: `/v3/users` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:36.458317 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-571718ae-f98b-43d4-82c1-e609064eb77d None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 10:52:36.459324 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-571718ae-f98b-43d4-82c1-e609064eb77d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:52:36.460898 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-571718ae-f98b-43d4-82c1-e609064eb77d None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:52:36.482306 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 395/796] 10.222.0.40 () {56 vars in 1135 bytes} [Fri Aug 26 10:52:36 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 305 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 10:52:36.491187 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571718ae-f98b-43d4-82c1-e609064eb77d None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:52:36.546272 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992b7d82-eb09-4732-accc-11532c8bb5b7 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:52:36.572784 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:52:36.583558 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831d512f-44fe-4dea-bf26-ee536eaaae81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:52:36.584194 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831d512f-44fe-4dea-bf26-ee536eaaae81 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:36.584475 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831d512f-44fe-4dea-bf26-ee536eaaae81 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:36.584978 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831d512f-44fe-4dea-bf26-ee536eaaae81 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:36.587912 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:52:36.592729 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-831d512f-44fe-4dea-bf26-ee536eaaae81 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:52:36.594262 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-831d512f-44fe-4dea-bf26-ee536eaaae81 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:52:36.602626 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571718ae-f98b-43d4-82c1-e609064eb77d None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:52:36.606992 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application [None req-831d512f-44fe-4dea-bf26-ee536eaaae81 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 26 10:52:36.606992 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 10:52:36.606992 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 10:52:36.606992 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 10:52:36.606992 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 10:52:36.606992 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 10:52:36.606992 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 10:52:36.606992 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 10:52:36.606992 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 10:52:36.606992 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 10:52:36.606992 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 10:52:36.606992 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 10:52:36.606992 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 26 10:52:36.606992 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 26 10:52:36.606992 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 26 10:52:36.606992 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 26 10:52:36.608099 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 10:52:36.608099 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 10:52:36.608099 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 26 10:52:36.608099 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 10:52:36.608099 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 26 10:52:36.608099 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 10:52:36.608099 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 10:52:36.608099 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 10:52:36.608099 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 10:52:36.608099 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 10:52:36.608099 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 10:52:36.608099 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application with Lock( Aug 26 10:52:36.608099 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 10:52:36.608099 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._enter() Aug 26 10:52:36.608099 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 10:52:36.608099 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 10:52:36.608099 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 10:52:36.608099 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.creator() Aug 26 10:52:36.609698 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 10:52:36.609698 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application created_value = creator( Aug 26 10:52:36.609698 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 26 10:52:36.609698 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 26 10:52:36.609698 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 26 10:52:36.609698 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 26 10:52:36.609698 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 26 10:52:36.609698 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 26 10:52:36.609698 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 26 10:52:36.609698 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Aug 26 10:52:36.613501 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 402/797] 10.222.0.40 () {56 vars in 1117 bytes} [Fri Aug 26 10:52:36 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 10:52:36.614271 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831d512f-44fe-4dea-bf26-ee536eaaae81 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:52:36.643305 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c5ebca-af54-498e-99ab-f8a09dce9403 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:52:36.644188 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c5ebca-af54-498e-99ab-f8a09dce9403 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:36.644603 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c5ebca-af54-498e-99ab-f8a09dce9403 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:36.645009 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c5ebca-af54-498e-99ab-f8a09dce9403 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:36.655585 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c5ebca-af54-498e-99ab-f8a09dce9403 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:52:36.657748 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c5ebca-af54-498e-99ab-f8a09dce9403 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:52:36.672880 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application [None req-82c5ebca-af54-498e-99ab-f8a09dce9403 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 10:52:36.672880 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 10:52:36.672880 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 10:52:36.672880 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 10:52:36.672880 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 10:52:36.672880 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 10:52:36.672880 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 10:52:36.672880 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 10:52:36.672880 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 10:52:36.672880 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 10:52:36.672880 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 10:52:36.672880 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 10:52:36.672880 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 10:52:36.672880 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 10:52:36.672880 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 10:52:36.672880 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 10:52:36.673968 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 10:52:36.673968 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 10:52:36.673968 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 10:52:36.673968 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 10:52:36.673968 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 10:52:36.673968 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 10:52:36.673968 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 10:52:36.673968 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application with Lock( Aug 26 10:52:36.673968 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 10:52:36.673968 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._enter() Aug 26 10:52:36.673968 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 10:52:36.673968 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 10:52:36.673968 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 10:52:36.673968 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.creator() Aug 26 10:52:36.673968 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 10:52:36.673968 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application created_value = creator( Aug 26 10:52:36.673968 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 10:52:36.674924 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 10:52:36.674924 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 10:52:36.674924 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Aug 26 10:52:36.676585 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 396/798] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Aug 26 10:52:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 26 10:52:36.677637 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c5ebca-af54-498e-99ab-f8a09dce9403 None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:52:36.721065 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831d512f-44fe-4dea-bf26-ee536eaaae81 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:52:36.760730 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e4c079-daa3-41e3-bc1f-e3b0e41de46d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:52:36.761725 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e4c079-daa3-41e3-bc1f-e3b0e41de46d None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:36.762155 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e4c079-daa3-41e3-bc1f-e3b0e41de46d None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:36.762533 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e4c079-daa3-41e3-bc1f-e3b0e41de46d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:36.774450 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e4c079-daa3-41e3-bc1f-e3b0e41de46d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 10:52:36.779657 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application [None req-71e4c079-daa3-41e3-bc1f-e3b0e41de46d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 26 10:52:36.779657 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 10:52:36.779657 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 10:52:36.779657 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 10:52:36.779657 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 10:52:36.779657 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 10:52:36.779657 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 10:52:36.779657 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 10:52:36.779657 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 10:52:36.779657 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 10:52:36.779657 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 10:52:36.779657 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 10:52:36.779657 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 10:52:36.779657 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 10:52:36.779657 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 10:52:36.779657 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 10:52:36.780648 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 10:52:36.780648 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 10:52:36.780648 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 10:52:36.780648 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 10:52:36.780648 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 10:52:36.780648 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 10:52:36.780648 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 10:52:36.780648 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 10:52:36.780648 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 10:52:36.780648 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 10:52:36.780648 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 10:52:36.780648 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application with Lock( Aug 26 10:52:36.780648 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 10:52:36.780648 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._enter() Aug 26 10:52:36.780648 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 10:52:36.780648 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 10:52:36.783408 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 10:52:36.783408 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.creator() Aug 26 10:52:36.783408 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 10:52:36.783408 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application created_value = creator( Aug 26 10:52:36.783408 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 10:52:36.783408 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 10:52:36.783408 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 10:52:36.783408 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 10:52:36.783408 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 10:52:36.783408 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 10:52:36.783408 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 26 10:52:36.783408 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Aug 26 10:52:36.783408 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 403/799] 10.222.0.40 () {56 vars in 1096 bytes} [Fri Aug 26 10:52:36 2022] GET /identity/v3/roles/admin => generated 83 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 10:52:36.783408 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e4c079-daa3-41e3-bc1f-e3b0e41de46d None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:52:36.791757 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c5ebca-af54-498e-99ab-f8a09dce9403 None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:52:36.825809 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d679b70-0cf8-425d-a1f5-906a4d6c50f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:52:36.826365 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d679b70-0cf8-425d-a1f5-906a4d6c50f9 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:36.826641 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d679b70-0cf8-425d-a1f5-906a4d6c50f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:36.826907 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d679b70-0cf8-425d-a1f5-906a4d6c50f9 None admin] PATH_INFO: `/v3/users` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:36.827624 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d679b70-0cf8-425d-a1f5-906a4d6c50f9 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 10:52:36.827921 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d679b70-0cf8-425d-a1f5-906a4d6c50f9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:52:36.830088 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d679b70-0cf8-425d-a1f5-906a4d6c50f9 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:52:36.839440 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:52:36.840731 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:52:36.852906 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 397/800] 10.222.0.40 () {56 vars in 1149 bytes} [Fri Aug 26 10:52:36 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 10:52:36.852906 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9bbcb8-aa79-4a76-9619-0fa8a4036a4c None None] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:36.852906 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9bbcb8-aa79-4a76-9619-0fa8a4036a4c None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:36.852906 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9bbcb8-aa79-4a76-9619-0fa8a4036a4c None None] PATH_INFO: `/` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:36.852906 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 398/801] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 26 10:52:36 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 10:52:36.852906 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9bbcb8-aa79-4a76-9619-0fa8a4036a4c None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:52:36.909564 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e4c079-daa3-41e3-bc1f-e3b0e41de46d None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:52:36.946391 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2218ed33-da1c-4424-988a-d2cbd1238260 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:52:36.947053 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2218ed33-da1c-4424-988a-d2cbd1238260 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:36.947349 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2218ed33-da1c-4424-988a-d2cbd1238260 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:36.947616 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2218ed33-da1c-4424-988a-d2cbd1238260 None admin] PATH_INFO: `/v3/domains` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:36.948271 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2218ed33-da1c-4424-988a-d2cbd1238260 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 10:52:36.948574 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2218ed33-da1c-4424-988a-d2cbd1238260 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:52:36.949943 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2218ed33-da1c-4424-988a-d2cbd1238260 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:52:36.967283 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 404/802] 10.222.0.40 () {56 vars in 1109 bytes} [Fri Aug 26 10:52:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 10:52:36.967283 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2218ed33-da1c-4424-988a-d2cbd1238260 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:52:36.976009 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9bbcb8-aa79-4a76-9619-0fa8a4036a4c None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:52:37.015081 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebbf1f11-7c08-4439-bfae-904441ab261e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:52:37.015839 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbf1f11-7c08-4439-bfae-904441ab261e None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:37.016121 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbf1f11-7c08-4439-bfae-904441ab261e None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:37.016959 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbf1f11-7c08-4439-bfae-904441ab261e None admin] PATH_INFO: `/v3/roles` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:37.018055 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebbf1f11-7c08-4439-bfae-904441ab261e None admin] RBAC: Adding query filter params (name=admin) {{(pid=81423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 10:52:37.018363 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebbf1f11-7c08-4439-bfae-904441ab261e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:52:37.019726 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebbf1f11-7c08-4439-bfae-904441ab261e None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:52:37.028548 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 399/803] 10.222.0.40 () {56 vars in 1099 bytes} [Fri Aug 26 10:52:36 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 10:52:37.029785 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e75cade-f629-40a8-9e4a-c2d847d63ed8 None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:37.030094 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e75cade-f629-40a8-9e4a-c2d847d63ed8 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:37.030366 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e75cade-f629-40a8-9e4a-c2d847d63ed8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:37.048895 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-5e75cade-f629-40a8-9e4a-c2d847d63ed8 None None] MFA Rules not processed for user `ca155a48e2024c32af3fd2e19e5fac11`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 10:52:37.050098 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-5e75cade-f629-40a8-9e4a-c2d847d63ed8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 10:52:37.076935 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2218ed33-da1c-4424-988a-d2cbd1238260 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:52:37.085384 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:52:37.112650 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fedd31e5-ecdd-44d2-856a-5c8ec48a6e84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:52:37.113277 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedd31e5-ecdd-44d2-856a-5c8ec48a6e84 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:37.113549 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedd31e5-ecdd-44d2-856a-5c8ec48a6e84 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:37.113816 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedd31e5-ecdd-44d2-856a-5c8ec48a6e84 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:37.115066 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:52:37.124506 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fedd31e5-ecdd-44d2-856a-5c8ec48a6e84 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:52:37.126510 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fedd31e5-ecdd-44d2-856a-5c8ec48a6e84 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:52:37.128631 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 400/804] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 26 10:52:37 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 26 10:52:37.129478 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e75cade-f629-40a8-9e4a-c2d847d63ed8 None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:52:37.148776 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application [None req-fedd31e5-ecdd-44d2-856a-5c8ec48a6e84 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 10:52:37.148776 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 10:52:37.148776 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 10:52:37.148776 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 10:52:37.148776 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 10:52:37.148776 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 10:52:37.148776 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 10:52:37.148776 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 10:52:37.148776 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 10:52:37.148776 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 10:52:37.148776 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 10:52:37.148776 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 10:52:37.148776 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 10:52:37.148776 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 10:52:37.148776 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 10:52:37.148776 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 10:52:37.149762 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 10:52:37.149762 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 10:52:37.149762 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 10:52:37.149762 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 10:52:37.149762 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 10:52:37.149762 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 10:52:37.149762 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 10:52:37.149762 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application with Lock( Aug 26 10:52:37.149762 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 10:52:37.149762 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._enter() Aug 26 10:52:37.149762 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 10:52:37.149762 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 10:52:37.149762 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 10:52:37.149762 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.creator() Aug 26 10:52:37.149762 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 10:52:37.149762 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application created_value = creator( Aug 26 10:52:37.149762 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 10:52:37.150702 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 10:52:37.150702 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 10:52:37.150702 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Aug 26 10:52:37.156118 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 405/805] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Aug 26 10:52:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 10:52:37.156118 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fedd31e5-ecdd-44d2-856a-5c8ec48a6e84 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:52:37.249498 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e75cade-f629-40a8-9e4a-c2d847d63ed8 None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:52:37.286258 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fedd31e5-ecdd-44d2-856a-5c8ec48a6e84 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:52:37.298153 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34edd72e-47ac-4025-9c89-24faf563feee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:52:37.299341 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34edd72e-47ac-4025-9c89-24faf563feee None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:37.299800 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34edd72e-47ac-4025-9c89-24faf563feee None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:37.300245 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34edd72e-47ac-4025-9c89-24faf563feee None admin] PATH_INFO: `/v3/projects/service` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:37.309896 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34edd72e-47ac-4025-9c89-24faf563feee None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:52:37.312403 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34edd72e-47ac-4025-9c89-24faf563feee None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:52:37.322857 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3cce8d5-afa3-4a69-9d61-f666af7fffa1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:52:37.323570 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3cce8d5-afa3-4a69-9d61-f666af7fffa1 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:37.323937 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3cce8d5-afa3-4a69-9d61-f666af7fffa1 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:37.324285 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3cce8d5-afa3-4a69-9d61-f666af7fffa1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:37.328389 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application [None req-34edd72e-47ac-4025-9c89-24faf563feee None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 10:52:37.328389 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 10:52:37.328389 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 10:52:37.328389 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 10:52:37.328389 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 10:52:37.328389 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 10:52:37.328389 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 10:52:37.328389 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 10:52:37.328389 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 10:52:37.328389 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 10:52:37.328389 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 10:52:37.328389 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 10:52:37.328389 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 26 10:52:37.328389 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 26 10:52:37.328389 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 26 10:52:37.329812 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 26 10:52:37.329812 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 10:52:37.329812 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 10:52:37.329812 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 10:52:37.329812 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 10:52:37.329812 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 10:52:37.329812 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 10:52:37.329812 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 10:52:37.329812 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application with Lock( Aug 26 10:52:37.329812 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 10:52:37.329812 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._enter() Aug 26 10:52:37.329812 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 10:52:37.329812 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 10:52:37.329812 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 10:52:37.329812 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.creator() Aug 26 10:52:37.329812 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 10:52:37.329812 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application created_value = creator( Aug 26 10:52:37.330745 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 26 10:52:37.330745 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 26 10:52:37.330745 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 26 10:52:37.330745 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 26 10:52:37.330745 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 26 10:52:37.330745 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 26 10:52:37.330745 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 10:52:37.330745 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Aug 26 10:52:37.338253 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 401/806] 10.222.0.40 () {56 vars in 1111 bytes} [Fri Aug 26 10:52:37 2022] GET /identity/v3/projects/service => generated 88 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 10:52:37.338253 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34edd72e-47ac-4025-9c89-24faf563feee None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:52:37.343964 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3cce8d5-afa3-4a69-9d61-f666af7fffa1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:52:37.345441 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3cce8d5-afa3-4a69-9d61-f666af7fffa1 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:52:37.360616 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application [None req-d3cce8d5-afa3-4a69-9d61-f666af7fffa1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 10:52:37.360616 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 10:52:37.360616 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 10:52:37.360616 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 10:52:37.360616 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 10:52:37.360616 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 10:52:37.360616 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 10:52:37.360616 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 10:52:37.360616 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 10:52:37.360616 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 10:52:37.360616 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 10:52:37.360616 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 10:52:37.360616 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 10:52:37.360616 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 10:52:37.360616 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 10:52:37.360616 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 10:52:37.361580 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 10:52:37.361580 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 10:52:37.361580 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 10:52:37.361580 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 10:52:37.361580 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 10:52:37.361580 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 10:52:37.361580 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 10:52:37.361580 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application with Lock( Aug 26 10:52:37.361580 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 10:52:37.361580 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._enter() Aug 26 10:52:37.361580 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 10:52:37.361580 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 10:52:37.361580 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 10:52:37.361580 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.creator() Aug 26 10:52:37.361580 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 10:52:37.361580 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application created_value = creator( Aug 26 10:52:37.361580 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 10:52:37.362492 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 10:52:37.362492 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 10:52:37.362492 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Aug 26 10:52:37.366290 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 406/807] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Aug 26 10:52:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 10:52:37.367140 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3cce8d5-afa3-4a69-9d61-f666af7fffa1 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:52:37.381932 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:52:37.402806 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:52:37.453281 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34edd72e-47ac-4025-9c89-24faf563feee None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:52:37.475803 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3cce8d5-afa3-4a69-9d61-f666af7fffa1 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:52:37.487210 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13e7280-902c-4c2a-a34f-c23bf54f22e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:52:37.487810 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13e7280-902c-4c2a-a34f-c23bf54f22e3 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:37.488086 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13e7280-902c-4c2a-a34f-c23bf54f22e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:37.488356 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13e7280-902c-4c2a-a34f-c23bf54f22e3 None admin] PATH_INFO: `/v3/domains` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:37.488996 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a13e7280-902c-4c2a-a34f-c23bf54f22e3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 10:52:37.489295 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a13e7280-902c-4c2a-a34f-c23bf54f22e3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:52:37.490685 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a13e7280-902c-4c2a-a34f-c23bf54f22e3 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:52:37.504523 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 402/808] 10.222.0.40 () {56 vars in 1109 bytes} [Fri Aug 26 10:52:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 10:52:37.505311 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13e7280-902c-4c2a-a34f-c23bf54f22e3 None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:52:37.514840 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3936e4-d6c1-4bcc-9546-2dd825e3ab12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:52:37.515560 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3936e4-d6c1-4bcc-9546-2dd825e3ab12 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:37.515841 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3936e4-d6c1-4bcc-9546-2dd825e3ab12 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:37.516113 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3936e4-d6c1-4bcc-9546-2dd825e3ab12 None admin] PATH_INFO: `/v3/projects` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:37.517636 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d3936e4-d6c1-4bcc-9546-2dd825e3ab12 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 10:52:37.517636 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d3936e4-d6c1-4bcc-9546-2dd825e3ab12 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:52:37.519096 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d3936e4-d6c1-4bcc-9546-2dd825e3ab12 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:52:37.530836 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 407/809] 10.222.0.40 () {56 vars in 1148 bytes} [Fri Aug 26 10:52:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 10:52:37.532561 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252e3c14-9362-4735-bd35-422313208266 None None] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:37.532844 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252e3c14-9362-4735-bd35-422313208266 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:37.533268 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252e3c14-9362-4735-bd35-422313208266 None None] PATH_INFO: `/` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:37.538531 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 408/810] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 26 10:52:37 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 10:52:37.539293 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252e3c14-9362-4735-bd35-422313208266 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:52:37.605104 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13e7280-902c-4c2a-a34f-c23bf54f22e3 None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:52:37.607780 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:52:37.615935 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:52:37.640026 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6300d9-eebf-4c4c-af7b-45d0d0052eb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:52:37.640280 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252e3c14-9362-4735-bd35-422313208266 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:52:37.640624 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6300d9-eebf-4c4c-af7b-45d0d0052eb0 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:37.640897 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6300d9-eebf-4c4c-af7b-45d0d0052eb0 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:37.641164 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6300d9-eebf-4c4c-af7b-45d0d0052eb0 None admin] PATH_INFO: `/v3/domains` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:37.642372 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae6300d9-eebf-4c4c-af7b-45d0d0052eb0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 10:52:37.642667 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae6300d9-eebf-4c4c-af7b-45d0d0052eb0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:52:37.644013 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae6300d9-eebf-4c4c-af7b-45d0d0052eb0 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:52:37.655057 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 403/811] 10.222.0.40 () {56 vars in 1109 bytes} [Fri Aug 26 10:52:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 10:52:37.655843 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6300d9-eebf-4c4c-af7b-45d0d0052eb0 None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:52:37.676523 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda0f21b-1570-4662-9857-0b2514077595 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:52:37.677152 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda0f21b-1570-4662-9857-0b2514077595 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:37.677426 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda0f21b-1570-4662-9857-0b2514077595 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:37.677696 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda0f21b-1570-4662-9857-0b2514077595 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:37.678306 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda0f21b-1570-4662-9857-0b2514077595 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:52:37.679688 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda0f21b-1570-4662-9857-0b2514077595 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:52:37.688675 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application [None req-cda0f21b-1570-4662-9857-0b2514077595 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 26 10:52:37.688675 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 10:52:37.688675 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 26 10:52:37.688675 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 26 10:52:37.688675 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 26 10:52:37.688675 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 26 10:52:37.688675 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 26 10:52:37.688675 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 26 10:52:37.688675 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 26 10:52:37.688675 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Aug 26 10:52:37.688675 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 26 10:52:37.688675 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Aug 26 10:52:37.688675 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 10:52:37.688675 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 10:52:37.688675 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 10:52:37.688675 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 10:52:37.688675 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 10:52:37.689873 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 10:52:37.689873 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 10:52:37.689873 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 10:52:37.689873 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 10:52:37.689873 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 10:52:37.689873 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 10:52:37.689873 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 26 10:52:37.689873 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 26 10:52:37.689873 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 26 10:52:37.689873 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 26 10:52:37.689873 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 10:52:37.689873 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 10:52:37.689873 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 10:52:37.689873 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 10:52:37.689873 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 10:52:37.689873 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 10:52:37.690750 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 10:52:37.690750 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application with Lock( Aug 26 10:52:37.690750 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 10:52:37.690750 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._enter() Aug 26 10:52:37.690750 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 10:52:37.690750 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 10:52:37.690750 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 10:52:37.690750 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.creator() Aug 26 10:52:37.690750 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 10:52:37.690750 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application created_value = creator( Aug 26 10:52:37.690750 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 26 10:52:37.690750 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 26 10:52:37.690750 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 26 10:52:37.690750 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Aug 26 10:52:37.690750 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 409/812] 10.222.0.40 () {56 vars in 1129 bytes} [Fri Aug 26 10:52:37 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 10:52:37.691559 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda0f21b-1570-4662-9857-0b2514077595 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:52:37.764349 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6300d9-eebf-4c4c-af7b-45d0d0052eb0 None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:52:37.791929 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda0f21b-1570-4662-9857-0b2514077595 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:52:37.798903 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6fd007b-3334-4149-b3ab-f6c54562036d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:52:37.799524 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6fd007b-3334-4149-b3ab-f6c54562036d None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:37.799815 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6fd007b-3334-4149-b3ab-f6c54562036d None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:37.800075 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6fd007b-3334-4149-b3ab-f6c54562036d None admin] PATH_INFO: `/v3/projects/service` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:37.806945 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6fd007b-3334-4149-b3ab-f6c54562036d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:52:37.808377 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6fd007b-3334-4149-b3ab-f6c54562036d None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:52:37.819940 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application [None req-f6fd007b-3334-4149-b3ab-f6c54562036d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 10:52:37.819940 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 10:52:37.819940 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 10:52:37.819940 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 10:52:37.819940 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 10:52:37.819940 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 10:52:37.819940 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 10:52:37.819940 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 10:52:37.819940 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 10:52:37.819940 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 10:52:37.819940 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 10:52:37.819940 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 10:52:37.819940 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 26 10:52:37.819940 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 26 10:52:37.819940 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 26 10:52:37.820848 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 26 10:52:37.820848 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 10:52:37.820848 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 10:52:37.820848 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 10:52:37.820848 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 10:52:37.820848 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 10:52:37.820848 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 10:52:37.820848 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 10:52:37.820848 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application with Lock( Aug 26 10:52:37.820848 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 10:52:37.820848 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._enter() Aug 26 10:52:37.820848 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 10:52:37.820848 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 10:52:37.820848 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 10:52:37.820848 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.creator() Aug 26 10:52:37.820848 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 10:52:37.820848 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application created_value = creator( Aug 26 10:52:37.822161 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 26 10:52:37.822161 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 26 10:52:37.822161 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 26 10:52:37.822161 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 26 10:52:37.822161 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 26 10:52:37.822161 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 26 10:52:37.822161 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 10:52:37.822161 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Aug 26 10:52:37.822161 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 404/813] 10.222.0.40 () {56 vars in 1111 bytes} [Fri Aug 26 10:52:37 2022] GET /identity/v3/projects/service => generated 88 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 10:52:37.822690 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac32b55-ae52-4a01-83fd-65e69f039335 None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:37.822841 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac32b55-ae52-4a01-83fd-65e69f039335 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:37.823065 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac32b55-ae52-4a01-83fd-65e69f039335 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:37.831748 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2f86f4-5e18-46af-b948-00d1a7f15863 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:52:37.832406 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2f86f4-5e18-46af-b948-00d1a7f15863 None admin] REQUEST_METHOD: `PUT` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:37.832684 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2f86f4-5e18-46af-b948-00d1a7f15863 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:37.832953 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2f86f4-5e18-46af-b948-00d1a7f15863 None admin] PATH_INFO: `/v3/projects/0e30ccf0c1204727b786876c141dbcd2/users/6529bea296c9482a973ced5fe9f2212f/roles/1f97e4602e2e4d6498913a81f9ae8d6a` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:37.834893 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd2f86f4-5e18-46af-b948-00d1a7f15863 None admin] RBAC: Authorizing `identity:create_grant(project_id=0e30ccf0c1204727b786876c141dbcd2, user_id=6529bea296c9482a973ced5fe9f2212f, role_id=1f97e4602e2e4d6498913a81f9ae8d6a)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:52:37.836322 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd2f86f4-5e18-46af-b948-00d1a7f15863 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:52:37.838556 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:52:37.840764 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-fac32b55-ae52-4a01-83fd-65e69f039335 None None] MFA Rules not processed for user `ca155a48e2024c32af3fd2e19e5fac11`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 10:52:37.841951 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-fac32b55-ae52-4a01-83fd-65e69f039335 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 10:52:37.848719 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 410/814] 10.222.0.40 () {58 vars in 1439 bytes} [Fri Aug 26 10:52:37 2022] PUT /identity/v3/projects/0e30ccf0c1204727b786876c141dbcd2/users/6529bea296c9482a973ced5fe9f2212f/roles/1f97e4602e2e4d6498913a81f9ae8d6a => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 10:52:37.849513 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2f86f4-5e18-46af-b948-00d1a7f15863 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:52:37.885636 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:52:37.934265 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 405/815] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 26 10:52:37 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 26 10:52:37.945805 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac32b55-ae52-4a01-83fd-65e69f039335 None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:52:37.972926 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2f86f4-5e18-46af-b948-00d1a7f15863 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:52:38.015694 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a64f50-65ac-4bf0-923e-ef1f0e383507 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:52:38.016880 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a64f50-65ac-4bf0-923e-ef1f0e383507 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:38.017618 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a64f50-65ac-4bf0-923e-ef1f0e383507 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:38.018038 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a64f50-65ac-4bf0-923e-ef1f0e383507 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:38.028944 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a64f50-65ac-4bf0-923e-ef1f0e383507 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:52:38.031145 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a64f50-65ac-4bf0-923e-ef1f0e383507 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:52:38.046495 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application [None req-70a64f50-65ac-4bf0-923e-ef1f0e383507 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 26 10:52:38.046495 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 10:52:38.046495 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 10:52:38.046495 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 10:52:38.046495 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 10:52:38.046495 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 10:52:38.046495 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 10:52:38.046495 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 10:52:38.046495 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 10:52:38.046495 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 10:52:38.046495 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 10:52:38.046495 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 10:52:38.046495 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 26 10:52:38.046495 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 26 10:52:38.046495 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 26 10:52:38.046495 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 26 10:52:38.048244 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 10:52:38.048244 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 10:52:38.048244 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 26 10:52:38.048244 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 10:52:38.048244 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 26 10:52:38.048244 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 10:52:38.048244 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 10:52:38.048244 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 10:52:38.048244 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 10:52:38.048244 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 10:52:38.048244 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 10:52:38.048244 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application with Lock( Aug 26 10:52:38.048244 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 10:52:38.048244 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._enter() Aug 26 10:52:38.048244 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 10:52:38.048244 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 10:52:38.048244 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 10:52:38.048244 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.creator() Aug 26 10:52:38.049640 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 10:52:38.049640 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application created_value = creator( Aug 26 10:52:38.049640 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 26 10:52:38.049640 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 26 10:52:38.049640 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 26 10:52:38.049640 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 26 10:52:38.049640 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 26 10:52:38.049640 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 26 10:52:38.049640 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 26 10:52:38.049640 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Aug 26 10:52:38.051302 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 411/816] 10.222.0.40 () {56 vars in 1093 bytes} [Fri Aug 26 10:52:37 2022] GET /identity/v3/users/nova => generated 82 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 10:52:38.052631 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a64f50-65ac-4bf0-923e-ef1f0e383507 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:52:38.089126 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac32b55-ae52-4a01-83fd-65e69f039335 None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:52:38.118673 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:52:38.138529 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d769dae0-22ce-4709-8fad-4d80a6107be8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:52:38.139357 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d769dae0-22ce-4709-8fad-4d80a6107be8 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:38.139784 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d769dae0-22ce-4709-8fad-4d80a6107be8 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:38.140199 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d769dae0-22ce-4709-8fad-4d80a6107be8 None admin] PATH_INFO: `/v3/services` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:38.141113 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d769dae0-22ce-4709-8fad-4d80a6107be8 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 10:52:38.141592 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d769dae0-22ce-4709-8fad-4d80a6107be8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:52:38.143192 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d769dae0-22ce-4709-8fad-4d80a6107be8 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:52:38.148724 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 406/817] 10.222.0.40 () {56 vars in 1124 bytes} [Fri Aug 26 10:52:37 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 10:52:38.154503 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d769dae0-22ce-4709-8fad-4d80a6107be8 None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:52:38.170080 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a64f50-65ac-4bf0-923e-ef1f0e383507 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:52:38.183504 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:52:38.207126 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7b6c5d-f0a6-41bb-abff-cbc58432a7fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:52:38.207740 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7b6c5d-f0a6-41bb-abff-cbc58432a7fd None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:38.208017 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7b6c5d-f0a6-41bb-abff-cbc58432a7fd None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:38.208352 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7b6c5d-f0a6-41bb-abff-cbc58432a7fd None admin] PATH_INFO: `/v3/projects` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:38.209050 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b7b6c5d-f0a6-41bb-abff-cbc58432a7fd None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 10:52:38.209702 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b7b6c5d-f0a6-41bb-abff-cbc58432a7fd None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:52:38.210862 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b7b6c5d-f0a6-41bb-abff-cbc58432a7fd None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:52:38.224536 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 412/818] 10.222.0.40 () {56 vars in 1148 bytes} [Fri Aug 26 10:52:38 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 10:52:38.225374 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7b6c5d-f0a6-41bb-abff-cbc58432a7fd None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:52:38.277618 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d769dae0-22ce-4709-8fad-4d80a6107be8 None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:52:38.314523 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ce8616-8246-4dea-ac15-66cd7ff31b8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:52:38.315410 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ce8616-8246-4dea-ac15-66cd7ff31b8c None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:38.315687 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ce8616-8246-4dea-ac15-66cd7ff31b8c None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:38.315959 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ce8616-8246-4dea-ac15-66cd7ff31b8c None admin] PATH_INFO: `/v3/users` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:38.316838 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10ce8616-8246-4dea-ac15-66cd7ff31b8c None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 10:52:38.317441 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10ce8616-8246-4dea-ac15-66cd7ff31b8c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:52:38.318903 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10ce8616-8246-4dea-ac15-66cd7ff31b8c None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:52:38.335579 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7b6c5d-f0a6-41bb-abff-cbc58432a7fd None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:52:38.336970 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 407/819] 10.222.0.40 () {56 vars in 1133 bytes} [Fri Aug 26 10:52:38 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 10:52:38.337620 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ce8616-8246-4dea-ac15-66cd7ff31b8c None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:52:38.356771 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:52:38.379661 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12c7d04-897f-4b0a-9c43-7b757717cbc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:52:38.380295 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12c7d04-897f-4b0a-9c43-7b757717cbc6 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:38.380574 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12c7d04-897f-4b0a-9c43-7b757717cbc6 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:38.380841 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12c7d04-897f-4b0a-9c43-7b757717cbc6 None admin] PATH_INFO: `/v3/services` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:38.381480 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e12c7d04-897f-4b0a-9c43-7b757717cbc6 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 10:52:38.381783 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e12c7d04-897f-4b0a-9c43-7b757717cbc6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:52:38.383185 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e12c7d04-897f-4b0a-9c43-7b757717cbc6 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:52:38.387985 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 413/820] 10.222.0.40 () {56 vars in 1124 bytes} [Fri Aug 26 10:52:38 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 10:52:38.388787 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12c7d04-897f-4b0a-9c43-7b757717cbc6 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:52:38.433825 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:52:38.454435 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ce8616-8246-4dea-ac15-66cd7ff31b8c None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:52:38.495971 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa830e2a-c5b1-4cf3-88e9-0fe6cbc99005 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:52:38.496633 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa830e2a-c5b1-4cf3-88e9-0fe6cbc99005 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:38.496911 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa830e2a-c5b1-4cf3-88e9-0fe6cbc99005 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:38.497340 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa830e2a-c5b1-4cf3-88e9-0fe6cbc99005 None admin] PATH_INFO: `/v3/services/network` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:38.497759 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa830e2a-c5b1-4cf3-88e9-0fe6cbc99005 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:52:38.499158 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa830e2a-c5b1-4cf3-88e9-0fe6cbc99005 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:52:38.500113 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12c7d04-897f-4b0a-9c43-7b757717cbc6 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:52:38.508961 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application [None req-fa830e2a-c5b1-4cf3-88e9-0fe6cbc99005 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 26 10:52:38.508961 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 10:52:38.508961 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 26 10:52:38.508961 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 26 10:52:38.508961 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 26 10:52:38.508961 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 26 10:52:38.508961 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 26 10:52:38.508961 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 26 10:52:38.508961 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 26 10:52:38.508961 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Aug 26 10:52:38.508961 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 26 10:52:38.508961 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Aug 26 10:52:38.508961 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 10:52:38.508961 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 10:52:38.508961 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 10:52:38.508961 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 10:52:38.508961 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 10:52:38.509961 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 10:52:38.509961 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 10:52:38.509961 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 10:52:38.509961 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 10:52:38.509961 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 10:52:38.509961 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 10:52:38.509961 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 26 10:52:38.509961 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 26 10:52:38.509961 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 26 10:52:38.509961 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 26 10:52:38.509961 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 10:52:38.509961 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 10:52:38.509961 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 10:52:38.509961 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 10:52:38.509961 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 10:52:38.509961 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 10:52:38.510860 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 10:52:38.510860 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application with Lock( Aug 26 10:52:38.510860 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 10:52:38.510860 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._enter() Aug 26 10:52:38.510860 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 10:52:38.510860 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 10:52:38.510860 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 10:52:38.510860 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.creator() Aug 26 10:52:38.510860 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 10:52:38.510860 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application created_value = creator( Aug 26 10:52:38.510860 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 26 10:52:38.510860 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 26 10:52:38.510860 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 26 10:52:38.510860 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Aug 26 10:52:38.510860 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 408/821] 10.222.0.40 () {56 vars in 1111 bytes} [Fri Aug 26 10:52:38 2022] GET /identity/v3/services/network => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 10:52:38.511690 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa830e2a-c5b1-4cf3-88e9-0fe6cbc99005 None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:52:38.540457 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec7d4a2-f527-4277-96d7-4b1b33af0198 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:52:38.541106 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec7d4a2-f527-4277-96d7-4b1b33af0198 None admin] REQUEST_METHOD: `PUT` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:38.541383 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec7d4a2-f527-4277-96d7-4b1b33af0198 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:38.541650 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec7d4a2-f527-4277-96d7-4b1b33af0198 None admin] PATH_INFO: `/v3/projects/0e30ccf0c1204727b786876c141dbcd2/users/50be12ccbd274a8792615e9dd10b61ca/roles/3bda770fcd5c46cdbfabcd7404a25e66` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:38.556233 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec7d4a2-f527-4277-96d7-4b1b33af0198 None admin] RBAC: Authorizing `identity:create_grant(project_id=0e30ccf0c1204727b786876c141dbcd2, user_id=50be12ccbd274a8792615e9dd10b61ca, role_id=3bda770fcd5c46cdbfabcd7404a25e66)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:52:38.557673 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec7d4a2-f527-4277-96d7-4b1b33af0198 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:52:38.582011 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 414/822] 10.222.0.40 () {58 vars in 1439 bytes} [Fri Aug 26 10:52:38 2022] PUT /identity/v3/projects/0e30ccf0c1204727b786876c141dbcd2/users/50be12ccbd274a8792615e9dd10b61ca/roles/3bda770fcd5c46cdbfabcd7404a25e66 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 10:52:38.582011 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec7d4a2-f527-4277-96d7-4b1b33af0198 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:52:38.614321 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:52:38.646458 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa830e2a-c5b1-4cf3-88e9-0fe6cbc99005 None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:52:38.694005 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10e2f8d-55e5-4f77-94b2-df297547b841 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:52:38.695062 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10e2f8d-55e5-4f77-94b2-df297547b841 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:38.695339 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10e2f8d-55e5-4f77-94b2-df297547b841 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:38.695605 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10e2f8d-55e5-4f77-94b2-df297547b841 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:38.712296 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec7d4a2-f527-4277-96d7-4b1b33af0198 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:52:38.718157 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:52:38.762397 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6894b673-6268-4602-8173-9c5e9ed41cf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:52:38.763520 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6894b673-6268-4602-8173-9c5e9ed41cf4 None admin] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:38.763796 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6894b673-6268-4602-8173-9c5e9ed41cf4 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:38.764238 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6894b673-6268-4602-8173-9c5e9ed41cf4 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:38.764910 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6894b673-6268-4602-8173-9c5e9ed41cf4 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:52:38.766782 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6894b673-6268-4602-8173-9c5e9ed41cf4 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:52:38.786487 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f10e2f8d-55e5-4f77-94b2-df297547b841 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:52:38.787949 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f10e2f8d-55e5-4f77-94b2-df297547b841 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:52:38.799099 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 415/823] 10.222.0.40 () {60 vars in 1144 bytes} [Fri Aug 26 10:52:38 2022] POST /identity/v3/endpoints => generated 346 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 10:52:38.800701 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6894b673-6268-4602-8173-9c5e9ed41cf4 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:52:38.909515 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:52:38.933638 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6894b673-6268-4602-8173-9c5e9ed41cf4 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:52:38.975811 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df2552d-ad1c-4a0e-bead-9993ef9af621 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:52:38.976653 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df2552d-ad1c-4a0e-bead-9993ef9af621 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:38.977031 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df2552d-ad1c-4a0e-bead-9993ef9af621 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:38.977401 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df2552d-ad1c-4a0e-bead-9993ef9af621 None admin] PATH_INFO: `/v3/services` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:38.978222 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df2552d-ad1c-4a0e-bead-9993ef9af621 None admin] RBAC: Adding query filter params (name=network) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 10:52:38.978626 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df2552d-ad1c-4a0e-bead-9993ef9af621 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:52:38.981090 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df2552d-ad1c-4a0e-bead-9993ef9af621 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:52:38.992187 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 416/824] 10.222.0.40 () {56 vars in 1112 bytes} [Fri Aug 26 10:52:38 2022] GET /identity/v3/services?name=network => generated 124 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 10:52:38.993589 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df2552d-ad1c-4a0e-bead-9993ef9af621 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:52:39.062771 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 409/825] 10.222.0.40 () {58 vars in 1282 bytes} [Fri Aug 26 10:52:38 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 552 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 26 10:52:39.070269 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10e2f8d-55e5-4f77-94b2-df297547b841 None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:52:39.118815 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df2552d-ad1c-4a0e-bead-9993ef9af621 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:52:39.128418 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:52:39.161415 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100a35fc-4e59-4108-9d57-f06ff3685b40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:52:39.161942 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100a35fc-4e59-4108-9d57-f06ff3685b40 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:39.162309 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100a35fc-4e59-4108-9d57-f06ff3685b40 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:39.162587 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100a35fc-4e59-4108-9d57-f06ff3685b40 None admin] PATH_INFO: `/v3/services` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:39.163223 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-100a35fc-4e59-4108-9d57-f06ff3685b40 None admin] RBAC: Adding query filter params (type=network) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 10:52:39.163572 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-100a35fc-4e59-4108-9d57-f06ff3685b40 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:52:39.164947 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-100a35fc-4e59-4108-9d57-f06ff3685b40 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:52:39.171383 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 417/826] 10.222.0.40 () {56 vars in 1112 bytes} [Fri Aug 26 10:52:38 2022] GET /identity/v3/services?type=network => generated 350 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 10:52:39.176909 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100a35fc-4e59-4108-9d57-f06ff3685b40 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:52:39.187861 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:52:39.191627 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10e2f8d-55e5-4f77-94b2-df297547b841 None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:52:39.233612 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785c6536-8e79-44ff-9e04-62c4b33450d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:52:39.234701 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785c6536-8e79-44ff-9e04-62c4b33450d3 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:39.235009 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785c6536-8e79-44ff-9e04-62c4b33450d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:39.235590 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785c6536-8e79-44ff-9e04-62c4b33450d3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:39.249458 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-785c6536-8e79-44ff-9e04-62c4b33450d3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:52:39.251702 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-785c6536-8e79-44ff-9e04-62c4b33450d3 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:52:39.270989 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application [None req-785c6536-8e79-44ff-9e04-62c4b33450d3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 10:52:39.270989 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 10:52:39.270989 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 10:52:39.270989 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 10:52:39.270989 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 10:52:39.270989 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 10:52:39.270989 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 10:52:39.270989 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 10:52:39.270989 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 10:52:39.270989 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 10:52:39.270989 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 10:52:39.270989 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 10:52:39.270989 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 10:52:39.270989 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 10:52:39.270989 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 10:52:39.270989 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 10:52:39.271981 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 10:52:39.271981 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 10:52:39.271981 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 10:52:39.271981 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 10:52:39.271981 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 10:52:39.271981 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 10:52:39.271981 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 10:52:39.271981 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application with Lock( Aug 26 10:52:39.271981 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 10:52:39.271981 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._enter() Aug 26 10:52:39.271981 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 10:52:39.271981 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 10:52:39.271981 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 10:52:39.271981 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.creator() Aug 26 10:52:39.271981 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 10:52:39.271981 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application created_value = creator( Aug 26 10:52:39.271981 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 10:52:39.272877 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 10:52:39.272877 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 10:52:39.272877 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Aug 26 10:52:39.272877 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 410/827] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Aug 26 10:52:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 10:52:39.277953 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785c6536-8e79-44ff-9e04-62c4b33450d3 None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:52:39.297415 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100a35fc-4e59-4108-9d57-f06ff3685b40 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:52:39.337128 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87482eb8-cd5c-4d50-9c61-05437edd44b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:52:39.338199 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87482eb8-cd5c-4d50-9c61-05437edd44b5 None admin] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:39.338680 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87482eb8-cd5c-4d50-9c61-05437edd44b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:39.339148 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87482eb8-cd5c-4d50-9c61-05437edd44b5 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:39.340029 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87482eb8-cd5c-4d50-9c61-05437edd44b5 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:52:39.341776 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87482eb8-cd5c-4d50-9c61-05437edd44b5 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:52:39.369370 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 418/828] 10.222.0.40 () {60 vars in 1144 bytes} [Fri Aug 26 10:52:39 2022] POST /identity/v3/endpoints => generated 337 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 26 10:52:39.466642 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:52:39.473005 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785c6536-8e79-44ff-9e04-62c4b33450d3 None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:52:39.528854 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028d98d8-f910-489d-a8ad-b849dd4158db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:52:39.529846 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028d98d8-f910-489d-a8ad-b849dd4158db None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:39.530350 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028d98d8-f910-489d-a8ad-b849dd4158db None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:39.530817 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028d98d8-f910-489d-a8ad-b849dd4158db None admin] PATH_INFO: `/v3/domains` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:39.531667 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-028d98d8-f910-489d-a8ad-b849dd4158db None admin] RBAC: Adding query filter params (name=Default) {{(pid=81423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 10:52:39.532214 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-028d98d8-f910-489d-a8ad-b849dd4158db None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:52:39.533999 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-028d98d8-f910-489d-a8ad-b849dd4158db None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:52:39.548502 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 411/829] 10.222.0.40 () {56 vars in 1109 bytes} [Fri Aug 26 10:52:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 10:52:39.554426 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87482eb8-cd5c-4d50-9c61-05437edd44b5 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:52:39.641031 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:52:39.693377 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87482eb8-cd5c-4d50-9c61-05437edd44b5 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:52:39.727729 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67056ffc-ec4b-41e2-a58e-7cb881bffb38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:52:39.728287 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67056ffc-ec4b-41e2-a58e-7cb881bffb38 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:39.728563 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67056ffc-ec4b-41e2-a58e-7cb881bffb38 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:39.728832 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67056ffc-ec4b-41e2-a58e-7cb881bffb38 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:39.736835 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67056ffc-ec4b-41e2-a58e-7cb881bffb38 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:52:39.738312 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67056ffc-ec4b-41e2-a58e-7cb881bffb38 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:52:39.750064 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application [None req-67056ffc-ec4b-41e2-a58e-7cb881bffb38 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 10:52:39.750064 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 10:52:39.750064 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 10:52:39.750064 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 10:52:39.750064 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 10:52:39.750064 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 10:52:39.750064 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 10:52:39.750064 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 10:52:39.750064 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 10:52:39.750064 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 10:52:39.750064 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 10:52:39.750064 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 10:52:39.750064 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 26 10:52:39.750064 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 26 10:52:39.750064 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 26 10:52:39.751654 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 26 10:52:39.751654 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 10:52:39.751654 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 10:52:39.751654 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 10:52:39.751654 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 10:52:39.751654 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 10:52:39.751654 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 10:52:39.751654 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 10:52:39.751654 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application with Lock( Aug 26 10:52:39.751654 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 10:52:39.751654 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._enter() Aug 26 10:52:39.751654 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 10:52:39.751654 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 10:52:39.751654 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 10:52:39.751654 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.creator() Aug 26 10:52:39.751654 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 10:52:39.751654 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application created_value = creator( Aug 26 10:52:39.752548 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 26 10:52:39.752548 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 26 10:52:39.752548 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 26 10:52:39.752548 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 26 10:52:39.752548 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 26 10:52:39.752548 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 26 10:52:39.752548 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 10:52:39.752548 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Aug 26 10:52:39.752548 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 419/830] 10.222.0.40 () {56 vars in 1111 bytes} [Fri Aug 26 10:52:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 10:52:39.759482 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028d98d8-f910-489d-a8ad-b849dd4158db None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:52:39.887500 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028d98d8-f910-489d-a8ad-b849dd4158db None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:52:39.924605 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c28a460-9555-4b4b-8c92-55856d77d16b None None] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:39.925186 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c28a460-9555-4b4b-8c92-55856d77d16b None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:39.925393 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c28a460-9555-4b4b-8c92-55856d77d16b None None] PATH_INFO: `/` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:39.926472 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8268a8f-e4b9-4d2f-8a6a-dc7451414fb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:52:39.927019 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 420/831] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 26 10:52:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 10:52:39.927192 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8268a8f-e4b9-4d2f-8a6a-dc7451414fb2 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:39.927468 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8268a8f-e4b9-4d2f-8a6a-dc7451414fb2 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:39.927736 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8268a8f-e4b9-4d2f-8a6a-dc7451414fb2 None admin] PATH_INFO: `/v3/projects` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:39.928567 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8268a8f-e4b9-4d2f-8a6a-dc7451414fb2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 10:52:39.928834 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8268a8f-e4b9-4d2f-8a6a-dc7451414fb2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:52:39.930271 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8268a8f-e4b9-4d2f-8a6a-dc7451414fb2 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:52:39.939384 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7faca0-7862-4b9c-b77c-91140fd84651 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:39.939669 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7faca0-7862-4b9c-b77c-91140fd84651 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:39.939944 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7faca0-7862-4b9c-b77c-91140fd84651 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:39.942760 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 412/832] 10.222.0.40 () {56 vars in 1148 bytes} [Fri Aug 26 10:52:39 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 10:52:39.948577 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8268a8f-e4b9-4d2f-8a6a-dc7451414fb2 None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:52:39.967109 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:52:39.969285 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-2f7faca0-7862-4b9c-b77c-91140fd84651 None None] MFA Rules not processed for user `ca155a48e2024c32af3fd2e19e5fac11`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 10:52:39.971236 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-2f7faca0-7862-4b9c-b77c-91140fd84651 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 10:52:40.057822 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 421/833] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 26 10:52:39 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 26 10:52:40.058976 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8268a8f-e4b9-4d2f-8a6a-dc7451414fb2 None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:52:40.092589 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b41199-58d5-4a98-bef9-c56b5071f722 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:52:40.093281 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b41199-58d5-4a98-bef9-c56b5071f722 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:40.093623 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b41199-58d5-4a98-bef9-c56b5071f722 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:40.093957 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b41199-58d5-4a98-bef9-c56b5071f722 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:40.095303 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b41199-58d5-4a98-bef9-c56b5071f722 None admin] RBAC: Adding query filter params (role.id=1f97e4602e2e4d6498913a81f9ae8d6a, scope.project.id=0e30ccf0c1204727b786876c141dbcd2, user.id=1f4b4a662db049ecacc865ecff10bdb0) {{(pid=81423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 10:52:40.095685 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b41199-58d5-4a98-bef9-c56b5071f722 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:52:40.097155 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b41199-58d5-4a98-bef9-c56b5071f722 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:52:40.103307 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 413/834] 10.222.0.40 () {56 vars in 1374 bytes} [Fri Aug 26 10:52:39 2022] GET /identity/v3/role_assignments?user.id=1f4b4a662db049ecacc865ecff10bdb0&scope.project.id=0e30ccf0c1204727b786876c141dbcd2&role.id=1f97e4602e2e4d6498913a81f9ae8d6a => generated 608 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 10:52:40.337202 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7faca0-7862-4b9c-b77c-91140fd84651 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:52:40.395105 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:52:40.441284 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7faca0-7862-4b9c-b77c-91140fd84651 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:52:40.475328 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4246f96e-5195-467b-9c9a-22a5c3780808 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:52:40.476143 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4246f96e-5195-467b-9c9a-22a5c3780808 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:40.476540 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4246f96e-5195-467b-9c9a-22a5c3780808 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:40.476904 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4246f96e-5195-467b-9c9a-22a5c3780808 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:40.488019 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4246f96e-5195-467b-9c9a-22a5c3780808 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 10:52:40.492760 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application [None req-4246f96e-5195-467b-9c9a-22a5c3780808 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 26 10:52:40.492760 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 10:52:40.492760 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 10:52:40.492760 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 10:52:40.492760 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 10:52:40.492760 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 10:52:40.492760 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 10:52:40.492760 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 10:52:40.492760 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 10:52:40.492760 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 10:52:40.492760 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 10:52:40.492760 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 10:52:40.492760 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 10:52:40.492760 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 10:52:40.492760 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 10:52:40.492760 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 10:52:40.493853 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 10:52:40.493853 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 10:52:40.493853 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 10:52:40.493853 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 10:52:40.493853 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 10:52:40.493853 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 10:52:40.493853 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 10:52:40.493853 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 10:52:40.493853 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 10:52:40.493853 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 10:52:40.493853 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 10:52:40.493853 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application with Lock( Aug 26 10:52:40.493853 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 10:52:40.493853 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._enter() Aug 26 10:52:40.493853 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 10:52:40.493853 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 10:52:40.494912 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 10:52:40.494912 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.creator() Aug 26 10:52:40.494912 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 10:52:40.494912 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application created_value = creator( Aug 26 10:52:40.494912 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 10:52:40.494912 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 10:52:40.494912 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 10:52:40.494912 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 10:52:40.494912 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 10:52:40.494912 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 10:52:40.494912 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 26 10:52:40.494912 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Aug 26 10:52:40.502126 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 422/835] 10.222.0.40 () {56 vars in 1096 bytes} [Fri Aug 26 10:52:40 2022] GET /identity/v3/roles/admin => generated 83 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 10:52:40.503467 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b41199-58d5-4a98-bef9-c56b5071f722 None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:52:40.588884 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:52:40.612992 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b41199-58d5-4a98-bef9-c56b5071f722 None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:52:40.647892 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30117623-492f-4c5f-baeb-545fee6094f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:52:40.648643 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30117623-492f-4c5f-baeb-545fee6094f5 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:40.649031 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30117623-492f-4c5f-baeb-545fee6094f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:40.649396 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30117623-492f-4c5f-baeb-545fee6094f5 None admin] PATH_INFO: `/v3/roles` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:40.650180 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30117623-492f-4c5f-baeb-545fee6094f5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 10:52:40.650579 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30117623-492f-4c5f-baeb-545fee6094f5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:52:40.652027 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30117623-492f-4c5f-baeb-545fee6094f5 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:52:40.665934 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 414/836] 10.222.0.40 () {56 vars in 1099 bytes} [Fri Aug 26 10:52:40 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 10:52:40.667270 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4246f96e-5195-467b-9c9a-22a5c3780808 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:52:40.769769 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4246f96e-5195-467b-9c9a-22a5c3780808 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:52:40.784840 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:52:40.786085 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda6a28f-a78d-4533-86e9-3e0bfb248ee3 None None] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:40.786406 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda6a28f-a78d-4533-86e9-3e0bfb248ee3 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:40.786644 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda6a28f-a78d-4533-86e9-3e0bfb248ee3 None None] PATH_INFO: `/` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:40.792113 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 415/837] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 26 10:52:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 26 10:52:40.793496 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9594d64-1ef6-401b-8ef0-7560202647c9 None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:40.793838 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9594d64-1ef6-401b-8ef0-7560202647c9 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:40.794207 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9594d64-1ef6-401b-8ef0-7560202647c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:40.809691 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d844fd9b-2418-401d-a757-f1aeac8be8df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:52:40.810628 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d844fd9b-2418-401d-a757-f1aeac8be8df None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:40.811119 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d844fd9b-2418-401d-a757-f1aeac8be8df None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:40.811570 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d844fd9b-2418-401d-a757-f1aeac8be8df None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:40.814084 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-d9594d64-1ef6-401b-8ef0-7560202647c9 None None] MFA Rules not processed for user `ca155a48e2024c32af3fd2e19e5fac11`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 10:52:40.815461 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-d9594d64-1ef6-401b-8ef0-7560202647c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 10:52:40.823512 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d844fd9b-2418-401d-a757-f1aeac8be8df None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:52:40.825313 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d844fd9b-2418-401d-a757-f1aeac8be8df None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:52:40.847123 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application [None req-d844fd9b-2418-401d-a757-f1aeac8be8df None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 10:52:40.847123 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 10:52:40.847123 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 10:52:40.847123 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 10:52:40.847123 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 10:52:40.847123 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 10:52:40.847123 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 10:52:40.847123 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 10:52:40.847123 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 10:52:40.847123 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 10:52:40.847123 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 10:52:40.847123 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 10:52:40.847123 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 10:52:40.847123 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 10:52:40.847123 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 10:52:40.847123 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 10:52:40.848304 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 10:52:40.848304 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 10:52:40.848304 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 10:52:40.848304 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 10:52:40.848304 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 10:52:40.848304 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 10:52:40.848304 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 10:52:40.848304 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application with Lock( Aug 26 10:52:40.848304 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 10:52:40.848304 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._enter() Aug 26 10:52:40.848304 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 10:52:40.848304 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 10:52:40.848304 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 10:52:40.848304 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.creator() Aug 26 10:52:40.848304 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 10:52:40.848304 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application created_value = creator( Aug 26 10:52:40.848304 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 10:52:40.849421 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 10:52:40.849421 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 10:52:40.849421 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Aug 26 10:52:40.851500 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 423/838] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Aug 26 10:52:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 10:52:40.857264 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d844fd9b-2418-401d-a757-f1aeac8be8df None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:52:40.906009 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 416/839] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 26 10:52:40 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 26 10:52:40.979607 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d844fd9b-2418-401d-a757-f1aeac8be8df None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:52:41.015013 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f73e2d-e778-45cf-8232-87fe304b42af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:52:41.015626 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f73e2d-e778-45cf-8232-87fe304b42af None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:41.015905 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f73e2d-e778-45cf-8232-87fe304b42af None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:41.016708 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f73e2d-e778-45cf-8232-87fe304b42af None admin] PATH_INFO: `/v3/domains` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:41.017668 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f73e2d-e778-45cf-8232-87fe304b42af None admin] RBAC: Adding query filter params (name=Default) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 10:52:41.017990 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f73e2d-e778-45cf-8232-87fe304b42af None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:52:41.019423 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f73e2d-e778-45cf-8232-87fe304b42af None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:52:41.032449 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 424/840] 10.222.0.40 () {56 vars in 1109 bytes} [Fri Aug 26 10:52:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 10:52:41.039190 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9594d64-1ef6-401b-8ef0-7560202647c9 None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:52:41.112766 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:52:41.146367 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9594d64-1ef6-401b-8ef0-7560202647c9 None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:52:41.181919 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d472f91-5406-4df2-ae2d-56c2b2248081 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:52:41.182960 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d472f91-5406-4df2-ae2d-56c2b2248081 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:41.183434 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d472f91-5406-4df2-ae2d-56c2b2248081 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:41.183892 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d472f91-5406-4df2-ae2d-56c2b2248081 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:41.187423 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f73e2d-e778-45cf-8232-87fe304b42af None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:52:41.192096 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d472f91-5406-4df2-ae2d-56c2b2248081 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:52:41.193801 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d472f91-5406-4df2-ae2d-56c2b2248081 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:52:41.207084 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application [None req-7d472f91-5406-4df2-ae2d-56c2b2248081 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 26 10:52:41.207084 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 10:52:41.207084 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 10:52:41.207084 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 10:52:41.207084 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 10:52:41.207084 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 10:52:41.207084 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 10:52:41.207084 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 10:52:41.207084 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 10:52:41.207084 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 10:52:41.207084 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 10:52:41.207084 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 10:52:41.207084 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 26 10:52:41.207084 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 26 10:52:41.207084 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 26 10:52:41.207084 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 26 10:52:41.208105 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 10:52:41.208105 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 10:52:41.208105 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 26 10:52:41.208105 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 10:52:41.208105 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 26 10:52:41.208105 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 10:52:41.208105 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 10:52:41.208105 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 10:52:41.208105 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 10:52:41.208105 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 10:52:41.208105 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 10:52:41.208105 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application with Lock( Aug 26 10:52:41.208105 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 10:52:41.208105 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._enter() Aug 26 10:52:41.208105 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 10:52:41.208105 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 10:52:41.208105 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 10:52:41.208105 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.creator() Aug 26 10:52:41.209125 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 10:52:41.209125 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application created_value = creator( Aug 26 10:52:41.209125 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 26 10:52:41.209125 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 26 10:52:41.209125 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 26 10:52:41.209125 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 26 10:52:41.209125 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 26 10:52:41.209125 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 26 10:52:41.209125 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 26 10:52:41.209125 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Aug 26 10:52:41.216342 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 417/841] 10.222.0.40 () {56 vars in 1096 bytes} [Fri Aug 26 10:52:41 2022] GET /identity/v3/users/swift => generated 83 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 10:52:41.218014 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d472f91-5406-4df2-ae2d-56c2b2248081 None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:52:41.220893 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:52:41.316242 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f73e2d-e778-45cf-8232-87fe304b42af None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:52:41.333070 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d472f91-5406-4df2-ae2d-56c2b2248081 None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:52:41.350919 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc7bc1e-da77-4974-af50-95801f326af7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:52:41.351554 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc7bc1e-da77-4974-af50-95801f326af7 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:41.351828 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc7bc1e-da77-4974-af50-95801f326af7 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:41.352100 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc7bc1e-da77-4974-af50-95801f326af7 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:41.363782 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc7bc1e-da77-4974-af50-95801f326af7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 10:52:41.364738 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:52:41.369433 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application [None req-8fc7bc1e-da77-4974-af50-95801f326af7 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 26 10:52:41.369433 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 10:52:41.369433 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 10:52:41.369433 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 10:52:41.369433 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 10:52:41.369433 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 10:52:41.369433 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 10:52:41.369433 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 10:52:41.369433 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 10:52:41.369433 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 10:52:41.369433 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 10:52:41.369433 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 10:52:41.369433 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 10:52:41.369433 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 10:52:41.369433 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 10:52:41.369433 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 10:52:41.370478 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 10:52:41.370478 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 10:52:41.370478 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 10:52:41.370478 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 10:52:41.370478 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 10:52:41.370478 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 10:52:41.370478 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 10:52:41.370478 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 10:52:41.370478 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 10:52:41.370478 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 10:52:41.370478 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 10:52:41.370478 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application with Lock( Aug 26 10:52:41.370478 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 10:52:41.370478 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._enter() Aug 26 10:52:41.370478 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 10:52:41.370478 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 10:52:41.371306 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 10:52:41.371306 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.creator() Aug 26 10:52:41.371306 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 10:52:41.371306 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application created_value = creator( Aug 26 10:52:41.371306 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 10:52:41.371306 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 10:52:41.371306 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 10:52:41.371306 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 10:52:41.371306 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 10:52:41.371306 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 10:52:41.371306 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 26 10:52:41.371306 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Aug 26 10:52:41.371306 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 425/842] 10.222.0.40 () {56 vars in 1102 bytes} [Fri Aug 26 10:52:41 2022] GET /identity/v3/roles/service => generated 85 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 26 10:52:41.372577 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d0f6ce-ef33-4742-a2d3-a4f2692397ad None None] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:41.372577 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d0f6ce-ef33-4742-a2d3-a4f2692397ad None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:41.372736 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d0f6ce-ef33-4742-a2d3-a4f2692397ad None None] PATH_INFO: `/` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:41.377913 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 426/843] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 26 10:52:41 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 10:52:41.378079 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f56ed70-b40f-4b50-9fd2-1d8e2b9dd2d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:52:41.378079 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f56ed70-b40f-4b50-9fd2-1d8e2b9dd2d6 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:41.378079 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f56ed70-b40f-4b50-9fd2-1d8e2b9dd2d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:41.378079 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f56ed70-b40f-4b50-9fd2-1d8e2b9dd2d6 None admin] PATH_INFO: `/v3/users` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:41.378820 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f56ed70-b40f-4b50-9fd2-1d8e2b9dd2d6 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 10:52:41.379121 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f56ed70-b40f-4b50-9fd2-1d8e2b9dd2d6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:52:41.379218 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148e2acf-a832-4a56-8672-e7b77813269c None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:41.379488 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148e2acf-a832-4a56-8672-e7b77813269c None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:41.379764 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148e2acf-a832-4a56-8672-e7b77813269c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:41.380530 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f56ed70-b40f-4b50-9fd2-1d8e2b9dd2d6 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:52:41.402882 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 418/844] 10.222.0.40 () {56 vars in 1135 bytes} [Fri Aug 26 10:52:41 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 10:52:41.402882 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f56ed70-b40f-4b50-9fd2-1d8e2b9dd2d6 None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:52:41.403597 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-148e2acf-a832-4a56-8672-e7b77813269c None None] MFA Rules not processed for user `ca155a48e2024c32af3fd2e19e5fac11`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 10:52:41.404975 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-148e2acf-a832-4a56-8672-e7b77813269c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 10:52:41.481925 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 427/845] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 26 10:52:41 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 26 10:52:41.483163 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148e2acf-a832-4a56-8672-e7b77813269c None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:52:41.503278 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:52:41.518489 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f56ed70-b40f-4b50-9fd2-1d8e2b9dd2d6 None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:52:41.565872 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fffb0ec2-60a4-458c-a5cf-9e8ac18949d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:52:41.566551 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffb0ec2-60a4-458c-a5cf-9e8ac18949d0 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:41.566829 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffb0ec2-60a4-458c-a5cf-9e8ac18949d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:41.567101 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffb0ec2-60a4-458c-a5cf-9e8ac18949d0 None admin] PATH_INFO: `/v3/roles` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:41.567772 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fffb0ec2-60a4-458c-a5cf-9e8ac18949d0 None admin] RBAC: Adding query filter params (name=service) {{(pid=81423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 10:52:41.568074 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fffb0ec2-60a4-458c-a5cf-9e8ac18949d0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:52:41.569435 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fffb0ec2-60a4-458c-a5cf-9e8ac18949d0 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:52:41.584775 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 419/846] 10.222.0.40 () {56 vars in 1103 bytes} [Fri Aug 26 10:52:41 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 10:52:41.586995 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fffb0ec2-60a4-458c-a5cf-9e8ac18949d0 None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:52:41.607918 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148e2acf-a832-4a56-8672-e7b77813269c None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:52:41.645565 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:52:41.654738 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6467dcd1-80d7-4bc1-9e62-4419237a49b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:52:41.655417 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6467dcd1-80d7-4bc1-9e62-4419237a49b8 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:41.655905 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6467dcd1-80d7-4bc1-9e62-4419237a49b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:41.656281 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6467dcd1-80d7-4bc1-9e62-4419237a49b8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:41.707440 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fffb0ec2-60a4-458c-a5cf-9e8ac18949d0 None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:52:41.748558 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2203a1-055d-4ad2-8e0b-3cb8b7827b94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:52:41.749480 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6467dcd1-80d7-4bc1-9e62-4419237a49b8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:52:41.749832 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2203a1-055d-4ad2-8e0b-3cb8b7827b94 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:41.750331 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2203a1-055d-4ad2-8e0b-3cb8b7827b94 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:41.750888 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2203a1-055d-4ad2-8e0b-3cb8b7827b94 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:41.750995 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6467dcd1-80d7-4bc1-9e62-4419237a49b8 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:52:41.763248 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de2203a1-055d-4ad2-8e0b-3cb8b7827b94 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:52:41.764864 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de2203a1-055d-4ad2-8e0b-3cb8b7827b94 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:52:41.771306 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:52:41.780651 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application [None req-de2203a1-055d-4ad2-8e0b-3cb8b7827b94 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 10:52:41.780651 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 10:52:41.780651 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 10:52:41.780651 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 10:52:41.780651 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 10:52:41.780651 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 10:52:41.780651 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 10:52:41.780651 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 10:52:41.780651 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 10:52:41.780651 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 10:52:41.780651 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 10:52:41.780651 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 10:52:41.780651 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 10:52:41.780651 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 10:52:41.780651 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 10:52:41.780651 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 10:52:41.781882 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 10:52:41.781882 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 10:52:41.781882 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 10:52:41.781882 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 10:52:41.781882 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 10:52:41.781882 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 10:52:41.781882 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 10:52:41.781882 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application with Lock( Aug 26 10:52:41.781882 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 10:52:41.781882 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._enter() Aug 26 10:52:41.781882 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 10:52:41.781882 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 10:52:41.781882 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 10:52:41.781882 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.creator() Aug 26 10:52:41.781882 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 10:52:41.781882 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application created_value = creator( Aug 26 10:52:41.781882 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 10:52:41.784294 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 10:52:41.784294 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 10:52:41.784294 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Aug 26 10:52:41.784294 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 420/847] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Aug 26 10:52:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 10:52:41.784294 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2203a1-055d-4ad2-8e0b-3cb8b7827b94 None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:52:41.878404 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 428/848] 10.222.0.40 () {58 vars in 1282 bytes} [Fri Aug 26 10:52:41 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 395 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 26 10:52:41.879484 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6467dcd1-80d7-4bc1-9e62-4419237a49b8 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:52:41.913523 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2203a1-055d-4ad2-8e0b-3cb8b7827b94 None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:52:41.915161 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:52:41.954311 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ffe66b0-6553-4a2e-90b5-274e36fd859b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:52:41.955148 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffe66b0-6553-4a2e-90b5-274e36fd859b None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:41.955423 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffe66b0-6553-4a2e-90b5-274e36fd859b None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:41.955684 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffe66b0-6553-4a2e-90b5-274e36fd859b None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:41.956275 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ffe66b0-6553-4a2e-90b5-274e36fd859b None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:52:41.957629 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ffe66b0-6553-4a2e-90b5-274e36fd859b None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:52:41.967193 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application [None req-6ffe66b0-6553-4a2e-90b5-274e36fd859b None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 26 10:52:41.967193 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 10:52:41.967193 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 26 10:52:41.967193 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 26 10:52:41.967193 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 26 10:52:41.967193 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 26 10:52:41.967193 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 26 10:52:41.967193 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 26 10:52:41.967193 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 26 10:52:41.967193 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Aug 26 10:52:41.967193 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 26 10:52:41.967193 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Aug 26 10:52:41.967193 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 10:52:41.967193 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 10:52:41.967193 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 10:52:41.967193 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 10:52:41.967193 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 10:52:41.968229 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 10:52:41.968229 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 10:52:41.968229 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 10:52:41.968229 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 10:52:41.968229 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 10:52:41.968229 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 10:52:41.968229 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 26 10:52:41.968229 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 26 10:52:41.968229 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 26 10:52:41.968229 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 26 10:52:41.968229 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 10:52:41.968229 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 10:52:41.968229 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 10:52:41.968229 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 10:52:41.968229 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 10:52:41.968229 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 10:52:41.969106 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 10:52:41.969106 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application with Lock( Aug 26 10:52:41.969106 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 10:52:41.969106 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._enter() Aug 26 10:52:41.969106 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 10:52:41.969106 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 10:52:41.969106 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 10:52:41.969106 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.creator() Aug 26 10:52:41.969106 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 10:52:41.969106 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application created_value = creator( Aug 26 10:52:41.969106 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 26 10:52:41.969106 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 26 10:52:41.969106 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 26 10:52:41.969106 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Aug 26 10:52:41.969106 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 421/849] 10.222.0.40 () {56 vars in 1114 bytes} [Fri Aug 26 10:52:41 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 10:52:41.971770 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ffe66b0-6553-4a2e-90b5-274e36fd859b None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:52:42.011209 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6467dcd1-80d7-4bc1-9e62-4419237a49b8 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:52:42.030040 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:52:42.050319 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e48ff37-3129-405f-b4dc-ab475a8035e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:52:42.050933 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e48ff37-3129-405f-b4dc-ab475a8035e7 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:42.051277 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e48ff37-3129-405f-b4dc-ab475a8035e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:42.051558 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e48ff37-3129-405f-b4dc-ab475a8035e7 None admin] PATH_INFO: `/v3/domains` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:42.052237 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e48ff37-3129-405f-b4dc-ab475a8035e7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 10:52:42.052535 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e48ff37-3129-405f-b4dc-ab475a8035e7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:52:42.053943 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e48ff37-3129-405f-b4dc-ab475a8035e7 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:52:42.067360 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 429/850] 10.222.0.40 () {56 vars in 1109 bytes} [Fri Aug 26 10:52:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 10:52:42.068234 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e48ff37-3129-405f-b4dc-ab475a8035e7 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:52:42.090558 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ffe66b0-6553-4a2e-90b5-274e36fd859b None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:52:42.127933 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eed1f63-3366-492e-94fc-a8d0a05c0f53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:52:42.128569 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eed1f63-3366-492e-94fc-a8d0a05c0f53 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:42.128842 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eed1f63-3366-492e-94fc-a8d0a05c0f53 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:42.129108 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eed1f63-3366-492e-94fc-a8d0a05c0f53 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:42.140841 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eed1f63-3366-492e-94fc-a8d0a05c0f53 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:52:42.143177 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eed1f63-3366-492e-94fc-a8d0a05c0f53 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:52:42.157890 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application [None req-2eed1f63-3366-492e-94fc-a8d0a05c0f53 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 10:52:42.157890 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 10:52:42.157890 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 10:52:42.157890 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 10:52:42.157890 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 10:52:42.157890 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 10:52:42.157890 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 10:52:42.157890 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 10:52:42.157890 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 10:52:42.157890 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 10:52:42.157890 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 10:52:42.157890 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 10:52:42.157890 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 10:52:42.157890 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 10:52:42.157890 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 10:52:42.157890 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 10:52:42.159080 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 10:52:42.159080 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 10:52:42.159080 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 10:52:42.159080 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 10:52:42.159080 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 10:52:42.159080 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 10:52:42.159080 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 10:52:42.159080 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application with Lock( Aug 26 10:52:42.159080 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 10:52:42.159080 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._enter() Aug 26 10:52:42.159080 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 10:52:42.159080 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 10:52:42.159080 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 10:52:42.159080 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.creator() Aug 26 10:52:42.159080 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 10:52:42.159080 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application created_value = creator( Aug 26 10:52:42.159080 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 10:52:42.160231 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 10:52:42.160231 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 10:52:42.160231 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Aug 26 10:52:42.160231 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 422/851] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Aug 26 10:52:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 10:52:42.160505 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122ee4d6-4536-4a38-a6b5-defe6b91c088 None None] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:42.160585 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122ee4d6-4536-4a38-a6b5-defe6b91c088 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:42.160789 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122ee4d6-4536-4a38-a6b5-defe6b91c088 None None] PATH_INFO: `/` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:42.166105 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 423/852] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 26 10:52:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 10:52:42.166105 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122ee4d6-4536-4a38-a6b5-defe6b91c088 None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:52:42.178251 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e48ff37-3129-405f-b4dc-ab475a8035e7 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:52:42.187378 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:52:42.217143 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2566b584-3b70-4761-a8ea-cded5c19a6e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:52:42.217846 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2566b584-3b70-4761-a8ea-cded5c19a6e3 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:42.218158 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2566b584-3b70-4761-a8ea-cded5c19a6e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:42.218641 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2566b584-3b70-4761-a8ea-cded5c19a6e3 None admin] PATH_INFO: `/v3/services` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:42.219721 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2566b584-3b70-4761-a8ea-cded5c19a6e3 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 10:52:42.220219 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2566b584-3b70-4761-a8ea-cded5c19a6e3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:52:42.222609 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2566b584-3b70-4761-a8ea-cded5c19a6e3 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:52:42.228642 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 430/853] 10.222.0.40 () {56 vars in 1114 bytes} [Fri Aug 26 10:52:42 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 10:52:42.229461 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2566b584-3b70-4761-a8ea-cded5c19a6e3 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:52:42.277597 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:52:42.281938 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122ee4d6-4536-4a38-a6b5-defe6b91c088 None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:52:42.316840 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a7f7ac-9c46-4864-8608-16206ac48a05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:52:42.317460 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a7f7ac-9c46-4864-8608-16206ac48a05 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:42.317731 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a7f7ac-9c46-4864-8608-16206ac48a05 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:42.318022 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a7f7ac-9c46-4864-8608-16206ac48a05 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:42.325423 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a7f7ac-9c46-4864-8608-16206ac48a05 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:52:42.326896 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a7f7ac-9c46-4864-8608-16206ac48a05 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:52:42.338857 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application [None req-77a7f7ac-9c46-4864-8608-16206ac48a05 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 26 10:52:42.338857 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 10:52:42.338857 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 10:52:42.338857 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 10:52:42.338857 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 10:52:42.338857 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 10:52:42.338857 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 10:52:42.338857 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 10:52:42.338857 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 10:52:42.338857 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 10:52:42.338857 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 10:52:42.338857 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 10:52:42.338857 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 26 10:52:42.338857 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 26 10:52:42.338857 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 26 10:52:42.338857 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 26 10:52:42.340000 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 10:52:42.340000 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 10:52:42.340000 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 26 10:52:42.340000 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 10:52:42.340000 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 26 10:52:42.340000 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 10:52:42.340000 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 10:52:42.340000 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 10:52:42.340000 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 10:52:42.340000 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 10:52:42.340000 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 10:52:42.340000 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application with Lock( Aug 26 10:52:42.340000 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 10:52:42.340000 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._enter() Aug 26 10:52:42.340000 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 10:52:42.340000 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 10:52:42.340000 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 10:52:42.340000 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.creator() Aug 26 10:52:42.340964 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 10:52:42.340964 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application created_value = creator( Aug 26 10:52:42.340964 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 26 10:52:42.340964 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 26 10:52:42.340964 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 26 10:52:42.340964 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 26 10:52:42.340964 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 26 10:52:42.340964 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 26 10:52:42.340964 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 26 10:52:42.340964 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Aug 26 10:52:42.340964 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 424/854] 10.222.0.40 () {56 vars in 1117 bytes} [Fri Aug 26 10:52:42 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 10:52:42.342274 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ce48f5-0b1a-46dd-bbcb-280bccf02487 None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:42.342718 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ce48f5-0b1a-46dd-bbcb-280bccf02487 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:42.342997 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ce48f5-0b1a-46dd-bbcb-280bccf02487 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:42.347249 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2566b584-3b70-4761-a8ea-cded5c19a6e3 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:52:42.364570 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-86ce48f5-0b1a-46dd-bbcb-280bccf02487 None None] MFA Rules not processed for user `ca155a48e2024c32af3fd2e19e5fac11`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 10:52:42.365852 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-86ce48f5-0b1a-46dd-bbcb-280bccf02487 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 10:52:42.385483 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863da945-e5a9-441e-86ce-88f5686401c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:52:42.386132 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863da945-e5a9-441e-86ce-88f5686401c6 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:42.386403 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863da945-e5a9-441e-86ce-88f5686401c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:42.386666 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863da945-e5a9-441e-86ce-88f5686401c6 None admin] PATH_INFO: `/v3/domains` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:42.387299 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-863da945-e5a9-441e-86ce-88f5686401c6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 10:52:42.387593 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-863da945-e5a9-441e-86ce-88f5686401c6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:52:42.390028 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-863da945-e5a9-441e-86ce-88f5686401c6 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:52:42.403106 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 431/855] 10.222.0.40 () {56 vars in 1109 bytes} [Fri Aug 26 10:52:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 10:52:42.404080 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863da945-e5a9-441e-86ce-88f5686401c6 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:52:42.453818 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 425/856] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 26 10:52:42 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 26 10:52:42.453818 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ce48f5-0b1a-46dd-bbcb-280bccf02487 None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:52:42.458641 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:52:42.528959 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:52:42.534699 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863da945-e5a9-441e-86ce-88f5686401c6 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:52:42.561863 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ce48f5-0b1a-46dd-bbcb-280bccf02487 None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:52:42.576564 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1a7662-ae2e-4329-b3c2-2b6ee6f4e422 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:52:42.577170 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1a7662-ae2e-4329-b3c2-2b6ee6f4e422 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:42.577448 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1a7662-ae2e-4329-b3c2-2b6ee6f4e422 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:42.577822 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1a7662-ae2e-4329-b3c2-2b6ee6f4e422 None admin] PATH_INFO: `/v3/services` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:42.578758 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d1a7662-ae2e-4329-b3c2-2b6ee6f4e422 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 10:52:42.579150 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d1a7662-ae2e-4329-b3c2-2b6ee6f4e422 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:52:42.580688 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d1a7662-ae2e-4329-b3c2-2b6ee6f4e422 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:52:42.594040 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 432/857] 10.222.0.40 () {56 vars in 1114 bytes} [Fri Aug 26 10:52:42 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 10:52:42.594040 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1a7662-ae2e-4329-b3c2-2b6ee6f4e422 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:52:42.613601 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7199bffb-5859-45de-8a72-4abc4c75521c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:52:42.614384 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7199bffb-5859-45de-8a72-4abc4c75521c None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:42.614656 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7199bffb-5859-45de-8a72-4abc4c75521c None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:42.614918 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7199bffb-5859-45de-8a72-4abc4c75521c None admin] PATH_INFO: `/v3/users` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:42.615661 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7199bffb-5859-45de-8a72-4abc4c75521c None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 10:52:42.615959 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7199bffb-5859-45de-8a72-4abc4c75521c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:52:42.617657 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7199bffb-5859-45de-8a72-4abc4c75521c None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:52:42.646897 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 426/858] 10.222.0.40 () {56 vars in 1149 bytes} [Fri Aug 26 10:52:42 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 10:52:42.648363 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7199bffb-5859-45de-8a72-4abc4c75521c None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:52:42.739178 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:52:42.746049 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1a7662-ae2e-4329-b3c2-2b6ee6f4e422 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:52:42.779177 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7199bffb-5859-45de-8a72-4abc4c75521c None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:52:42.784397 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44cc8cf-e0e3-449a-9a1d-ba17f63450a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:52:42.784863 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44cc8cf-e0e3-449a-9a1d-ba17f63450a3 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:42.785142 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44cc8cf-e0e3-449a-9a1d-ba17f63450a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:42.786050 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44cc8cf-e0e3-449a-9a1d-ba17f63450a3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:42.793144 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c44cc8cf-e0e3-449a-9a1d-ba17f63450a3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:52:42.794653 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c44cc8cf-e0e3-449a-9a1d-ba17f63450a3 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:52:42.807029 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application [None req-c44cc8cf-e0e3-449a-9a1d-ba17f63450a3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 10:52:42.807029 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 10:52:42.807029 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 10:52:42.807029 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 10:52:42.807029 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 10:52:42.807029 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 10:52:42.807029 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 10:52:42.807029 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 10:52:42.807029 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 10:52:42.807029 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 10:52:42.807029 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 10:52:42.807029 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 10:52:42.807029 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 26 10:52:42.807029 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 26 10:52:42.807029 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 26 10:52:42.807978 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 26 10:52:42.807978 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 10:52:42.807978 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 10:52:42.807978 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 10:52:42.807978 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 10:52:42.807978 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 10:52:42.807978 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 10:52:42.807978 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 10:52:42.807978 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application with Lock( Aug 26 10:52:42.807978 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 10:52:42.807978 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._enter() Aug 26 10:52:42.807978 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 10:52:42.807978 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 10:52:42.807978 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 10:52:42.807978 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.creator() Aug 26 10:52:42.807978 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 10:52:42.807978 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application created_value = creator( Aug 26 10:52:42.808882 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 26 10:52:42.808882 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 26 10:52:42.808882 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 26 10:52:42.808882 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 26 10:52:42.808882 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 26 10:52:42.808882 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 26 10:52:42.808882 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 10:52:42.808882 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Aug 26 10:52:42.812937 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 433/859] 10.222.0.40 () {56 vars in 1111 bytes} [Fri Aug 26 10:52:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 10:52:42.813811 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44cc8cf-e0e3-449a-9a1d-ba17f63450a3 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:52:42.830821 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5df73b-f9a6-48d7-b56f-2e7851addaed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:52:42.831435 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5df73b-f9a6-48d7-b56f-2e7851addaed None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:42.831709 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5df73b-f9a6-48d7-b56f-2e7851addaed None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:42.831976 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5df73b-f9a6-48d7-b56f-2e7851addaed None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:42.839970 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:52:42.911604 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c5df73b-f9a6-48d7-b56f-2e7851addaed None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:52:42.913054 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c5df73b-f9a6-48d7-b56f-2e7851addaed None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:52:42.930223 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44cc8cf-e0e3-449a-9a1d-ba17f63450a3 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:52:42.964317 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a884bfe-4fae-4c90-b083-bc8e8ddb1045 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:52:42.965036 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a884bfe-4fae-4c90-b083-bc8e8ddb1045 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:42.965390 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a884bfe-4fae-4c90-b083-bc8e8ddb1045 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:42.965741 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a884bfe-4fae-4c90-b083-bc8e8ddb1045 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:42.966419 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a884bfe-4fae-4c90-b083-bc8e8ddb1045 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:52:42.967853 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a884bfe-4fae-4c90-b083-bc8e8ddb1045 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:52:42.977789 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application [None req-6a884bfe-4fae-4c90-b083-bc8e8ddb1045 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 26 10:52:42.977789 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 10:52:42.977789 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 26 10:52:42.977789 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 26 10:52:42.977789 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 26 10:52:42.977789 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 26 10:52:42.977789 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 26 10:52:42.977789 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 26 10:52:42.977789 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 26 10:52:42.977789 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Aug 26 10:52:42.977789 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 26 10:52:42.977789 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Aug 26 10:52:42.977789 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 10:52:42.977789 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 10:52:42.977789 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 10:52:42.977789 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 10:52:42.979034 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 10:52:42.979034 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 10:52:42.979034 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 10:52:42.979034 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 10:52:42.979034 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 10:52:42.979034 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 10:52:42.979034 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 10:52:42.979034 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 26 10:52:42.979034 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 26 10:52:42.979034 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 26 10:52:42.979034 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 26 10:52:42.979034 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 10:52:42.979034 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 10:52:42.979034 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 10:52:42.979034 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 10:52:42.979034 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 10:52:42.979034 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 10:52:42.980255 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 10:52:42.980255 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application with Lock( Aug 26 10:52:42.980255 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 10:52:42.980255 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._enter() Aug 26 10:52:42.980255 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 10:52:42.980255 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 10:52:42.980255 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 10:52:42.980255 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.creator() Aug 26 10:52:42.980255 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 10:52:42.980255 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application created_value = creator( Aug 26 10:52:42.980255 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 26 10:52:42.980255 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 26 10:52:42.980255 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 26 10:52:42.980255 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Aug 26 10:52:42.982047 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 434/860] 10.222.0.40 () {56 vars in 1132 bytes} [Fri Aug 26 10:52:42 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 10:52:42.982123 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:52:42.982197 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a884bfe-4fae-4c90-b083-bc8e8ddb1045 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:52:43.026223 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 427/861] 10.222.0.40 () {58 vars in 1282 bytes} [Fri Aug 26 10:52:42 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 26 10:52:43.027095 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5df73b-f9a6-48d7-b56f-2e7851addaed None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:52:43.083547 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:52:43.097823 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a884bfe-4fae-4c90-b083-bc8e8ddb1045 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:52:43.136064 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8bd6a6e-7279-4d81-a39b-4bde05871007 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:52:43.136768 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bd6a6e-7279-4d81-a39b-4bde05871007 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:43.137117 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bd6a6e-7279-4d81-a39b-4bde05871007 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:43.137457 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bd6a6e-7279-4d81-a39b-4bde05871007 None admin] PATH_INFO: `/v3/projects` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:43.137856 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5df73b-f9a6-48d7-b56f-2e7851addaed None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:52:43.138307 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8bd6a6e-7279-4d81-a39b-4bde05871007 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 10:52:43.138683 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8bd6a6e-7279-4d81-a39b-4bde05871007 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:52:43.141441 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8bd6a6e-7279-4d81-a39b-4bde05871007 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:52:43.161323 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 435/862] 10.222.0.40 () {56 vars in 1148 bytes} [Fri Aug 26 10:52:42 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 10:52:43.161323 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8bd6a6e-7279-4d81-a39b-4bde05871007 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:52:43.175043 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec02a2b-14f7-4da6-ab61-d5653e5435be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:52:43.175759 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec02a2b-14f7-4da6-ab61-d5653e5435be None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:43.176043 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec02a2b-14f7-4da6-ab61-d5653e5435be None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:43.176317 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec02a2b-14f7-4da6-ab61-d5653e5435be None admin] PATH_INFO: `/v3/services` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:43.176952 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec02a2b-14f7-4da6-ab61-d5653e5435be None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 10:52:43.177257 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec02a2b-14f7-4da6-ab61-d5653e5435be None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:52:43.178657 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec02a2b-14f7-4da6-ab61-d5653e5435be None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:52:43.183361 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 428/863] 10.222.0.40 () {56 vars in 1126 bytes} [Fri Aug 26 10:52:43 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 10:52:43.184140 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec02a2b-14f7-4da6-ab61-d5653e5435be None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:52:43.215674 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:52:43.287777 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8bd6a6e-7279-4d81-a39b-4bde05871007 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:52:43.292830 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec02a2b-14f7-4da6-ab61-d5653e5435be None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:52:43.325823 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b6b9399-950d-481a-94f7-b408bbb2a6bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:52:43.326293 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353357d6-352f-42ed-b26d-aeb661057130 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:52:43.326707 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6b9399-950d-481a-94f7-b408bbb2a6bb None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:43.326900 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353357d6-352f-42ed-b26d-aeb661057130 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:43.327178 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6b9399-950d-481a-94f7-b408bbb2a6bb None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:43.327272 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353357d6-352f-42ed-b26d-aeb661057130 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:43.327449 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353357d6-352f-42ed-b26d-aeb661057130 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:43.327711 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6b9399-950d-481a-94f7-b408bbb2a6bb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:43.328171 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-353357d6-352f-42ed-b26d-aeb661057130 None admin] RBAC: Adding query filter params (role.id=1f97e4602e2e4d6498913a81f9ae8d6a, scope.project.id=0e30ccf0c1204727b786876c141dbcd2, user.id=6529bea296c9482a973ced5fe9f2212f) {{(pid=81423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 10:52:43.328479 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-353357d6-352f-42ed-b26d-aeb661057130 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:52:43.330479 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-353357d6-352f-42ed-b26d-aeb661057130 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:52:43.336124 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 429/864] 10.222.0.40 () {56 vars in 1374 bytes} [Fri Aug 26 10:52:43 2022] GET /identity/v3/role_assignments?user.id=6529bea296c9482a973ced5fe9f2212f&scope.project.id=0e30ccf0c1204727b786876c141dbcd2&role.id=1f97e4602e2e4d6498913a81f9ae8d6a => generated 608 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 10:52:43.336906 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353357d6-352f-42ed-b26d-aeb661057130 None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:52:43.348573 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b6b9399-950d-481a-94f7-b408bbb2a6bb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:52:43.350882 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b6b9399-950d-481a-94f7-b408bbb2a6bb None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:52:43.363215 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:52:43.373596 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application [None req-2b6b9399-950d-481a-94f7-b408bbb2a6bb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 10:52:43.373596 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 10:52:43.373596 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 10:52:43.373596 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 10:52:43.373596 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 10:52:43.373596 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 10:52:43.373596 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 10:52:43.373596 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 10:52:43.373596 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 10:52:43.373596 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 10:52:43.373596 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 10:52:43.373596 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 10:52:43.373596 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 10:52:43.373596 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 10:52:43.373596 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 10:52:43.373596 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 10:52:43.374841 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 10:52:43.374841 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 10:52:43.374841 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 10:52:43.374841 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 10:52:43.374841 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 10:52:43.374841 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 10:52:43.374841 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 10:52:43.374841 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application with Lock( Aug 26 10:52:43.374841 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 10:52:43.374841 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._enter() Aug 26 10:52:43.374841 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 10:52:43.374841 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 10:52:43.374841 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 10:52:43.374841 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.creator() Aug 26 10:52:43.374841 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 10:52:43.374841 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application created_value = creator( Aug 26 10:52:43.374841 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 10:52:43.375930 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 10:52:43.375930 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 10:52:43.375930 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Aug 26 10:52:43.378331 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 436/865] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Aug 26 10:52:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 10:52:43.383708 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b6b9399-950d-481a-94f7-b408bbb2a6bb None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:52:43.468802 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:52:43.478903 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353357d6-352f-42ed-b26d-aeb661057130 None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:52:43.525796 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b6ad06-9320-4b88-9b85-b086ea4b2387 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:52:43.526775 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b6ad06-9320-4b88-9b85-b086ea4b2387 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:43.527114 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b6ad06-9320-4b88-9b85-b086ea4b2387 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:43.527453 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b6ad06-9320-4b88-9b85-b086ea4b2387 None admin] PATH_INFO: `/v3/services` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:43.528151 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b6ad06-9320-4b88-9b85-b086ea4b2387 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 10:52:43.530014 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b6ad06-9320-4b88-9b85-b086ea4b2387 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:52:43.530134 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b6b9399-950d-481a-94f7-b408bbb2a6bb None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:52:43.530719 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b6ad06-9320-4b88-9b85-b086ea4b2387 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:52:43.536941 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 430/866] 10.222.0.40 () {56 vars in 1126 bytes} [Fri Aug 26 10:52:43 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 10:52:43.574627 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e030823e-c04d-4a7d-9e5e-77c65303cfb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:52:43.575171 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e030823e-c04d-4a7d-9e5e-77c65303cfb6 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:43.575452 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e030823e-c04d-4a7d-9e5e-77c65303cfb6 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:43.575727 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e030823e-c04d-4a7d-9e5e-77c65303cfb6 None admin] PATH_INFO: `/v3/domains` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:43.576405 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e030823e-c04d-4a7d-9e5e-77c65303cfb6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 10:52:43.576698 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e030823e-c04d-4a7d-9e5e-77c65303cfb6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:52:43.578108 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e030823e-c04d-4a7d-9e5e-77c65303cfb6 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:52:43.593069 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 437/867] 10.222.0.40 () {56 vars in 1109 bytes} [Fri Aug 26 10:52:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 10:52:43.598897 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b6ad06-9320-4b88-9b85-b086ea4b2387 None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:52:43.716568 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b6ad06-9320-4b88-9b85-b086ea4b2387 None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:52:43.755201 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec8c7c7-76de-4706-a365-9d30d274ab8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:52:43.755986 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec8c7c7-76de-4706-a365-9d30d274ab8f None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:43.756355 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec8c7c7-76de-4706-a365-9d30d274ab8f None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:43.756723 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec8c7c7-76de-4706-a365-9d30d274ab8f None admin] PATH_INFO: `/v3/projects/service` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:43.763938 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec8c7c7-76de-4706-a365-9d30d274ab8f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:52:43.765482 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec8c7c7-76de-4706-a365-9d30d274ab8f None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:52:43.778011 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application [None req-8ec8c7c7-76de-4706-a365-9d30d274ab8f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 10:52:43.778011 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 10:52:43.778011 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 10:52:43.778011 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 10:52:43.778011 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 10:52:43.778011 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 10:52:43.778011 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 10:52:43.778011 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 10:52:43.778011 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 10:52:43.778011 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 10:52:43.778011 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 10:52:43.778011 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 10:52:43.778011 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 26 10:52:43.778011 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 26 10:52:43.778011 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 26 10:52:43.779077 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 26 10:52:43.779077 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 10:52:43.779077 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 10:52:43.779077 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 10:52:43.779077 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 10:52:43.779077 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 10:52:43.779077 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 10:52:43.779077 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 10:52:43.779077 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application with Lock( Aug 26 10:52:43.779077 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 10:52:43.779077 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._enter() Aug 26 10:52:43.779077 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 10:52:43.779077 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 10:52:43.779077 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 10:52:43.779077 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.creator() Aug 26 10:52:43.779077 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 10:52:43.779077 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application created_value = creator( Aug 26 10:52:43.780137 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 26 10:52:43.780137 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 26 10:52:43.780137 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 26 10:52:43.780137 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 26 10:52:43.780137 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 26 10:52:43.780137 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 26 10:52:43.780137 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 10:52:43.780137 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Aug 26 10:52:43.781855 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 431/868] 10.222.0.40 () {56 vars in 1111 bytes} [Fri Aug 26 10:52:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 10:52:43.788132 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e030823e-c04d-4a7d-9e5e-77c65303cfb6 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:52:43.852716 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:52:43.903114 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e030823e-c04d-4a7d-9e5e-77c65303cfb6 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:52:43.940446 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c27c14-b8e5-4c46-b44b-670055e7ef82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:52:43.941304 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c27c14-b8e5-4c46-b44b-670055e7ef82 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:43.941747 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c27c14-b8e5-4c46-b44b-670055e7ef82 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:43.942202 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c27c14-b8e5-4c46-b44b-670055e7ef82 None admin] PATH_INFO: `/v3/projects` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:43.943222 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c27c14-b8e5-4c46-b44b-670055e7ef82 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 10:52:43.943675 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c27c14-b8e5-4c46-b44b-670055e7ef82 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:52:43.945257 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c27c14-b8e5-4c46-b44b-670055e7ef82 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:52:43.958610 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 438/869] 10.222.0.40 () {56 vars in 1148 bytes} [Fri Aug 26 10:52:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 10:52:43.964662 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec8c7c7-76de-4706-a365-9d30d274ab8f None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:52:43.987375 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:52:44.076580 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec8c7c7-76de-4706-a365-9d30d274ab8f None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:52:44.113857 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ceb789-c0b2-4de8-bcff-3909b2b274a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:52:44.114417 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ceb789-c0b2-4de8-bcff-3909b2b274a5 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:44.114690 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ceb789-c0b2-4de8-bcff-3909b2b274a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:44.114957 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ceb789-c0b2-4de8-bcff-3909b2b274a5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:44.115682 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ceb789-c0b2-4de8-bcff-3909b2b274a5 None admin] RBAC: Adding query filter params (role.id=3bda770fcd5c46cdbfabcd7404a25e66, scope.project.id=0e30ccf0c1204727b786876c141dbcd2, user.id=50be12ccbd274a8792615e9dd10b61ca) {{(pid=81423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 10:52:44.115984 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ceb789-c0b2-4de8-bcff-3909b2b274a5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:52:44.117402 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ceb789-c0b2-4de8-bcff-3909b2b274a5 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:52:44.123937 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 432/870] 10.222.0.40 () {56 vars in 1374 bytes} [Fri Aug 26 10:52:43 2022] GET /identity/v3/role_assignments?user.id=50be12ccbd274a8792615e9dd10b61ca&scope.project.id=0e30ccf0c1204727b786876c141dbcd2&role.id=3bda770fcd5c46cdbfabcd7404a25e66 => generated 608 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 10:52:44.571896 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f96e38f-a566-4d81-95d7-11fdeadbddbb None None] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:44.572416 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f96e38f-a566-4d81-95d7-11fdeadbddbb None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:44.572416 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f96e38f-a566-4d81-95d7-11fdeadbddbb None None] PATH_INFO: `/` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:44.574053 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 439/871] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 26 10:52:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 10:52:44.580160 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072c1403-1cd8-4558-b5e6-957673b56f89 None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:44.580439 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072c1403-1cd8-4558-b5e6-957673b56f89 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:44.580710 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072c1403-1cd8-4558-b5e6-957673b56f89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:44.598951 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-072c1403-1cd8-4558-b5e6-957673b56f89 None None] MFA Rules not processed for user `ca155a48e2024c32af3fd2e19e5fac11`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 10:52:44.600145 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-072c1403-1cd8-4558-b5e6-957673b56f89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 10:52:44.669951 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 433/872] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 26 10:52:44 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 26 10:52:44.951147 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f96e38f-a566-4d81-95d7-11fdeadbddbb None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:52:45.064124 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f96e38f-a566-4d81-95d7-11fdeadbddbb None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:52:45.085111 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:52:45.100186 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62fb9cb6-781a-4617-8cd6-eb77dbd4635d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:52:45.101037 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fb9cb6-781a-4617-8cd6-eb77dbd4635d None admin] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:45.101315 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fb9cb6-781a-4617-8cd6-eb77dbd4635d None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:45.101589 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fb9cb6-781a-4617-8cd6-eb77dbd4635d None admin] PATH_INFO: `/v3/services` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:45.102323 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62fb9cb6-781a-4617-8cd6-eb77dbd4635d None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:52:45.103738 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62fb9cb6-781a-4617-8cd6-eb77dbd4635d None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:52:45.122124 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 440/873] 10.222.0.40 () {60 vars in 1141 bytes} [Fri Aug 26 10:52:44 2022] POST /identity/v3/services => generated 251 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 10:52:45.449867 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d10b913-2baf-4b75-a145-4013c66da50d None None] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:45.450228 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d10b913-2baf-4b75-a145-4013c66da50d None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:45.450444 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d10b913-2baf-4b75-a145-4013c66da50d None None] PATH_INFO: `/` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:45.451919 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 434/874] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 26 10:52:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 10:52:45.457862 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57ff48e-33eb-4198-ab3e-281ea2a00bee None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:45.458461 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57ff48e-33eb-4198-ab3e-281ea2a00bee None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:45.458931 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57ff48e-33eb-4198-ab3e-281ea2a00bee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:45.477940 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-b57ff48e-33eb-4198-ab3e-281ea2a00bee None None] MFA Rules not processed for user `ca155a48e2024c32af3fd2e19e5fac11`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 10:52:45.479501 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-b57ff48e-33eb-4198-ab3e-281ea2a00bee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 10:52:45.553655 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa9a05e-9fc8-4258-88f2-06eaf402641b None None] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:45.554001 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa9a05e-9fc8-4258-88f2-06eaf402641b None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:45.554342 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa9a05e-9fc8-4258-88f2-06eaf402641b None None] PATH_INFO: `/` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:45.554555 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 441/875] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 26 10:52:45 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 26 10:52:45.559012 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 435/876] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 26 10:52:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 26 10:52:45.560935 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952b78fe-f75c-4072-8618-7893d96b61b3 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:45.561223 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952b78fe-f75c-4072-8618-7893d96b61b3 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:45.561494 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952b78fe-f75c-4072-8618-7893d96b61b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:45.580991 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-952b78fe-f75c-4072-8618-7893d96b61b3 None None] MFA Rules not processed for user `ca155a48e2024c32af3fd2e19e5fac11`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 10:52:45.582277 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-952b78fe-f75c-4072-8618-7893d96b61b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 10:52:45.651025 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 442/877] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 26 10:52:45 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 26 10:52:45.832993 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa9a05e-9fc8-4258-88f2-06eaf402641b None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:52:45.849036 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:52:45.933039 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952b78fe-f75c-4072-8618-7893d96b61b3 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:52:45.957637 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa9a05e-9fc8-4258-88f2-06eaf402641b None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:52:45.979247 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:52:45.995854 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1a86ea-110e-47f8-b534-0ca47e1a6d87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:52:45.996518 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1a86ea-110e-47f8-b534-0ca47e1a6d87 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:45.996790 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1a86ea-110e-47f8-b534-0ca47e1a6d87 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:45.997053 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1a86ea-110e-47f8-b534-0ca47e1a6d87 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:45.997901 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f1a86ea-110e-47f8-b534-0ca47e1a6d87 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:52:45.999316 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f1a86ea-110e-47f8-b534-0ca47e1a6d87 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:52:46.011281 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application [None req-1f1a86ea-110e-47f8-b534-0ca47e1a6d87 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 26 10:52:46.011281 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 10:52:46.011281 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 26 10:52:46.011281 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 26 10:52:46.011281 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 26 10:52:46.011281 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 26 10:52:46.011281 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 26 10:52:46.011281 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 26 10:52:46.011281 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 26 10:52:46.011281 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Aug 26 10:52:46.011281 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 26 10:52:46.011281 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Aug 26 10:52:46.011281 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 10:52:46.011281 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 10:52:46.011281 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 10:52:46.011281 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 10:52:46.011281 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 10:52:46.012311 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 10:52:46.012311 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 10:52:46.012311 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 10:52:46.012311 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 10:52:46.012311 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 10:52:46.012311 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 10:52:46.012311 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 26 10:52:46.012311 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 26 10:52:46.012311 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 26 10:52:46.012311 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 26 10:52:46.012311 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 10:52:46.012311 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 10:52:46.012311 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 10:52:46.012311 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 10:52:46.012311 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 10:52:46.012311 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 10:52:46.013134 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 10:52:46.013134 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application with Lock( Aug 26 10:52:46.013134 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 10:52:46.013134 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._enter() Aug 26 10:52:46.013134 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 10:52:46.013134 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 10:52:46.013134 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 10:52:46.013134 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.creator() Aug 26 10:52:46.013134 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 10:52:46.013134 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application created_value = creator( Aug 26 10:52:46.013134 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 26 10:52:46.013134 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 26 10:52:46.013134 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 26 10:52:46.013134 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Aug 26 10:52:46.013134 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 436/878] 10.222.0.40 () {56 vars in 1126 bytes} [Fri Aug 26 10:52:45 2022] GET /identity/v3/services/object-store => generated 93 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 10:52:46.021074 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1a86ea-110e-47f8-b534-0ca47e1a6d87 None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:52:46.045524 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952b78fe-f75c-4072-8618-7893d96b61b3 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:52:46.082892 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd695b5f-f85f-4652-8648-acb543ca6619 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:52:46.083899 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd695b5f-f85f-4652-8648-acb543ca6619 None admin] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:46.084487 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd695b5f-f85f-4652-8648-acb543ca6619 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:46.084824 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd695b5f-f85f-4652-8648-acb543ca6619 None admin] PATH_INFO: `/v3/services` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:46.085716 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd695b5f-f85f-4652-8648-acb543ca6619 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:52:46.087172 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd695b5f-f85f-4652-8648-acb543ca6619 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:52:46.110025 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 443/879] 10.222.0.40 () {60 vars in 1141 bytes} [Fri Aug 26 10:52:45 2022] POST /identity/v3/services => generated 269 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 10:52:46.112986 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:52:46.142536 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1a86ea-110e-47f8-b534-0ca47e1a6d87 None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:52:46.189868 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0d783c-2dd7-42a4-baaf-aa3771b5f9bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:52:46.190523 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0d783c-2dd7-42a4-baaf-aa3771b5f9bc None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:46.190799 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0d783c-2dd7-42a4-baaf-aa3771b5f9bc None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:46.191067 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0d783c-2dd7-42a4-baaf-aa3771b5f9bc None admin] PATH_INFO: `/v3/services` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:46.191698 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f0d783c-2dd7-42a4-baaf-aa3771b5f9bc None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 10:52:46.191992 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f0d783c-2dd7-42a4-baaf-aa3771b5f9bc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:52:46.193456 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f0d783c-2dd7-42a4-baaf-aa3771b5f9bc None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:52:46.199185 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 437/880] 10.222.0.40 () {56 vars in 1122 bytes} [Fri Aug 26 10:52:46 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 10:52:46.205575 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd695b5f-f85f-4652-8648-acb543ca6619 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:52:46.273068 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d002e129-2f6c-483f-931e-775d0f38472a None None] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:46.273368 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d002e129-2f6c-483f-931e-775d0f38472a None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:46.273605 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d002e129-2f6c-483f-931e-775d0f38472a None None] PATH_INFO: `/` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:46.275407 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 438/881] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 26 10:52:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 26 10:52:46.280237 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2b84d4-6f8c-4b5e-8ee4-5d5b447b7263 None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:46.280525 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2b84d4-6f8c-4b5e-8ee4-5d5b447b7263 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:46.280804 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2b84d4-6f8c-4b5e-8ee4-5d5b447b7263 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:46.301529 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-fa2b84d4-6f8c-4b5e-8ee4-5d5b447b7263 None None] MFA Rules not processed for user `ca155a48e2024c32af3fd2e19e5fac11`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 10:52:46.302771 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-fa2b84d4-6f8c-4b5e-8ee4-5d5b447b7263 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 10:52:46.315265 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:52:46.321740 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd695b5f-f85f-4652-8648-acb543ca6619 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:52:46.372886 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6602cf4-ab83-4bf5-a7ae-06e1299f4572 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:52:46.373566 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6602cf4-ab83-4bf5-a7ae-06e1299f4572 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:46.374066 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6602cf4-ab83-4bf5-a7ae-06e1299f4572 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:46.374346 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6602cf4-ab83-4bf5-a7ae-06e1299f4572 None admin] PATH_INFO: `/v3/services` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:46.375465 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6602cf4-ab83-4bf5-a7ae-06e1299f4572 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 10:52:46.375957 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6602cf4-ab83-4bf5-a7ae-06e1299f4572 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:52:46.377626 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6602cf4-ab83-4bf5-a7ae-06e1299f4572 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:52:46.383779 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 439/883] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 26 10:52:46 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 26 10:52:46.386043 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 444/883] 10.222.0.40 () {56 vars in 1122 bytes} [Fri Aug 26 10:52:46 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 10:52:46.654974 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2b84d4-6f8c-4b5e-8ee4-5d5b447b7263 None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:52:46.725353 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:52:46.763271 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2b84d4-6f8c-4b5e-8ee4-5d5b447b7263 None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:52:46.799368 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a133604-28d6-455f-8446-32f4649fdf1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:52:46.800001 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a133604-28d6-455f-8446-32f4649fdf1b None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:46.800280 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a133604-28d6-455f-8446-32f4649fdf1b None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:46.800554 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a133604-28d6-455f-8446-32f4649fdf1b None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:46.811377 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a133604-28d6-455f-8446-32f4649fdf1b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 10:52:46.816685 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application [None req-5a133604-28d6-455f-8446-32f4649fdf1b None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 26 10:52:46.816685 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 10:52:46.816685 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 10:52:46.816685 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 10:52:46.816685 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 10:52:46.816685 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 10:52:46.816685 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 10:52:46.816685 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 10:52:46.816685 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 10:52:46.816685 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 10:52:46.816685 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 10:52:46.816685 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 10:52:46.816685 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 10:52:46.816685 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 10:52:46.816685 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 10:52:46.816685 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 10:52:46.817603 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 10:52:46.817603 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 10:52:46.817603 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 10:52:46.817603 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 10:52:46.817603 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 10:52:46.817603 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 10:52:46.817603 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 10:52:46.817603 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 10:52:46.817603 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 10:52:46.817603 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 10:52:46.817603 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 10:52:46.817603 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application with Lock( Aug 26 10:52:46.817603 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 10:52:46.817603 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._enter() Aug 26 10:52:46.817603 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 10:52:46.817603 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 10:52:46.818462 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 10:52:46.818462 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.creator() Aug 26 10:52:46.818462 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 10:52:46.818462 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application created_value = creator( Aug 26 10:52:46.818462 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 10:52:46.818462 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 10:52:46.818462 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 10:52:46.818462 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 10:52:46.818462 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 10:52:46.818462 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 10:52:46.818462 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 26 10:52:46.818462 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Aug 26 10:52:46.822216 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 440/884] 10.222.0.40 () {56 vars in 1120 bytes} [Fri Aug 26 10:52:46 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 10:52:46.824254 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6602cf4-ab83-4bf5-a7ae-06e1299f4572 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:52:46.934167 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6602cf4-ab83-4bf5-a7ae-06e1299f4572 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:52:46.968969 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba94010-e313-48e5-837a-5bbbf4868304 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:52:46.969622 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba94010-e313-48e5-837a-5bbbf4868304 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:46.969900 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba94010-e313-48e5-837a-5bbbf4868304 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:46.970202 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba94010-e313-48e5-837a-5bbbf4868304 None admin] PATH_INFO: `/v3/roles` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:46.970952 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ba94010-e313-48e5-837a-5bbbf4868304 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 10:52:46.971253 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ba94010-e313-48e5-837a-5bbbf4868304 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:52:46.972615 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ba94010-e313-48e5-837a-5bbbf4868304 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:52:46.986109 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 445/885] 10.222.0.40 () {56 vars in 1115 bytes} [Fri Aug 26 10:52:46 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 10:52:46.987672 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a133604-28d6-455f-8446-32f4649fdf1b None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:52:47.099397 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a133604-28d6-455f-8446-32f4649fdf1b None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:52:47.134381 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:52:47.138598 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f95957f-6bcb-4435-9198-9fde780224be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:52:47.139355 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f95957f-6bcb-4435-9198-9fde780224be None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:47.139832 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f95957f-6bcb-4435-9198-9fde780224be None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:47.140120 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f95957f-6bcb-4435-9198-9fde780224be None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:47.151847 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f95957f-6bcb-4435-9198-9fde780224be None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:52:47.153302 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f95957f-6bcb-4435-9198-9fde780224be None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:52:47.168248 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application [None req-6f95957f-6bcb-4435-9198-9fde780224be None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 10:52:47.168248 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 10:52:47.168248 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 10:52:47.168248 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 10:52:47.168248 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 10:52:47.168248 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 10:52:47.168248 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 10:52:47.168248 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 10:52:47.168248 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 10:52:47.168248 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 10:52:47.168248 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 10:52:47.168248 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 10:52:47.168248 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 10:52:47.168248 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 10:52:47.168248 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 10:52:47.168248 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 10:52:47.170430 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 10:52:47.170430 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 10:52:47.170430 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 10:52:47.170430 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 10:52:47.170430 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 10:52:47.170430 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 10:52:47.170430 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 10:52:47.170430 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application with Lock( Aug 26 10:52:47.170430 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 10:52:47.170430 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._enter() Aug 26 10:52:47.170430 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 10:52:47.170430 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 10:52:47.170430 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 10:52:47.170430 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.creator() Aug 26 10:52:47.170430 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 10:52:47.170430 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application created_value = creator( Aug 26 10:52:47.170430 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 10:52:47.173209 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 10:52:47.173209 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 10:52:47.173209 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Aug 26 10:52:47.175232 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 441/886] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Aug 26 10:52:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 10:52:47.176328 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba94010-e313-48e5-837a-5bbbf4868304 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:52:47.206477 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:52:47.246986 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5b4853-25fb-4c9f-ab3c-a83bed1439b4 None None] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:47.247438 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5b4853-25fb-4c9f-ab3c-a83bed1439b4 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:47.247714 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5b4853-25fb-4c9f-ab3c-a83bed1439b4 None None] PATH_INFO: `/` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:47.250104 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 442/887] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 26 10:52:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 26 10:52:47.259061 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7bb6d7d-f987-4209-bc43-2932d8b51ec1 None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:47.259363 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7bb6d7d-f987-4209-bc43-2932d8b51ec1 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:47.259593 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7bb6d7d-f987-4209-bc43-2932d8b51ec1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:47.277711 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-f7bb6d7d-f987-4209-bc43-2932d8b51ec1 None None] MFA Rules not processed for user `ca155a48e2024c32af3fd2e19e5fac11`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 10:52:47.278947 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-f7bb6d7d-f987-4209-bc43-2932d8b51ec1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 10:52:47.294554 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba94010-e313-48e5-837a-5bbbf4868304 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:52:47.345202 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebec9db1-5478-466c-8c29-872cdfef484e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:52:47.346730 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebec9db1-5478-466c-8c29-872cdfef484e None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:47.347342 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebec9db1-5478-466c-8c29-872cdfef484e None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:47.347805 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebec9db1-5478-466c-8c29-872cdfef484e None admin] PATH_INFO: `/v3/domains` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:47.348902 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebec9db1-5478-466c-8c29-872cdfef484e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 10:52:47.349770 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebec9db1-5478-466c-8c29-872cdfef484e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:52:47.351636 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebec9db1-5478-466c-8c29-872cdfef484e None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:52:47.354370 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 443/888] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 26 10:52:47 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 26 10:52:47.370671 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 446/889] 10.222.0.40 () {56 vars in 1109 bytes} [Fri Aug 26 10:52:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 10:52:47.372031 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7bb6d7d-f987-4209-bc43-2932d8b51ec1 None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:52:47.477487 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7bb6d7d-f987-4209-bc43-2932d8b51ec1 None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:52:47.489988 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:52:47.515973 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ffd9f4c-87be-4784-88ca-75c51daea314 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:52:47.516937 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffd9f4c-87be-4784-88ca-75c51daea314 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:47.517412 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffd9f4c-87be-4784-88ca-75c51daea314 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:47.517903 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffd9f4c-87be-4784-88ca-75c51daea314 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:47.525254 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ffd9f4c-87be-4784-88ca-75c51daea314 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:52:47.526916 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ffd9f4c-87be-4784-88ca-75c51daea314 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:52:47.539351 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application [None req-9ffd9f4c-87be-4784-88ca-75c51daea314 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 26 10:52:47.539351 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 10:52:47.539351 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 10:52:47.539351 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 10:52:47.539351 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 10:52:47.539351 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 10:52:47.539351 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 10:52:47.539351 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 10:52:47.539351 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 10:52:47.539351 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 10:52:47.539351 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 10:52:47.539351 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 10:52:47.539351 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 26 10:52:47.539351 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 26 10:52:47.539351 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 26 10:52:47.539351 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 26 10:52:47.540358 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 10:52:47.540358 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 10:52:47.540358 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 26 10:52:47.540358 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 10:52:47.540358 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 26 10:52:47.540358 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 10:52:47.540358 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 10:52:47.540358 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 10:52:47.540358 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 10:52:47.540358 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 10:52:47.540358 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 10:52:47.540358 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application with Lock( Aug 26 10:52:47.540358 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 10:52:47.540358 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._enter() Aug 26 10:52:47.540358 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 10:52:47.540358 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 10:52:47.540358 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 10:52:47.540358 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.creator() Aug 26 10:52:47.541294 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 10:52:47.541294 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application created_value = creator( Aug 26 10:52:47.541294 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 26 10:52:47.541294 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 26 10:52:47.541294 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 26 10:52:47.541294 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 26 10:52:47.541294 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 26 10:52:47.541294 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 26 10:52:47.541294 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 26 10:52:47.541294 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Aug 26 10:52:47.543886 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 444/890] 10.222.0.40 () {56 vars in 1117 bytes} [Fri Aug 26 10:52:47 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 26 10:52:47.554062 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebec9db1-5478-466c-8c29-872cdfef484e None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:52:47.637323 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:52:47.657825 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebec9db1-5478-466c-8c29-872cdfef484e None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:52:47.682297 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ffd9f4c-87be-4784-88ca-75c51daea314 None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:52:47.696504 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084097cb-d566-4e1c-885f-15752391694f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:52:47.697125 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084097cb-d566-4e1c-885f-15752391694f None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:47.697408 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084097cb-d566-4e1c-885f-15752391694f None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:47.697704 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084097cb-d566-4e1c-885f-15752391694f None admin] PATH_INFO: `/v3/users` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:47.698734 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-084097cb-d566-4e1c-885f-15752391694f None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 10:52:47.699045 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-084097cb-d566-4e1c-885f-15752391694f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:52:47.700674 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-084097cb-d566-4e1c-885f-15752391694f None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:52:47.726068 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 447/891] 10.222.0.40 () {56 vars in 1149 bytes} [Fri Aug 26 10:52:47 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 10:52:47.730209 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084097cb-d566-4e1c-885f-15752391694f None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:52:47.795388 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ffd9f4c-87be-4784-88ca-75c51daea314 None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:52:47.835054 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0611ad3a-502a-46ec-8f16-9e4f49de3027 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:52:47.836129 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0611ad3a-502a-46ec-8f16-9e4f49de3027 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:47.836667 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0611ad3a-502a-46ec-8f16-9e4f49de3027 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:47.837189 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0611ad3a-502a-46ec-8f16-9e4f49de3027 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:47.837970 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0611ad3a-502a-46ec-8f16-9e4f49de3027 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:52:47.839612 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0611ad3a-502a-46ec-8f16-9e4f49de3027 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:52:47.849751 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application [None req-0611ad3a-502a-46ec-8f16-9e4f49de3027 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 26 10:52:47.849751 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 10:52:47.849751 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 26 10:52:47.849751 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 26 10:52:47.849751 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 26 10:52:47.849751 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 26 10:52:47.849751 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 26 10:52:47.849751 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 26 10:52:47.849751 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 26 10:52:47.849751 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Aug 26 10:52:47.849751 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 26 10:52:47.849751 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Aug 26 10:52:47.849751 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 10:52:47.849751 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 10:52:47.849751 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 10:52:47.849751 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 10:52:47.849751 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 10:52:47.850773 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 10:52:47.850773 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 10:52:47.850773 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 10:52:47.850773 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 10:52:47.850773 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 10:52:47.850773 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 10:52:47.850773 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 26 10:52:47.850773 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 26 10:52:47.850773 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 26 10:52:47.850773 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 26 10:52:47.850773 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 10:52:47.850773 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 10:52:47.850773 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 10:52:47.850773 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 10:52:47.850773 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 10:52:47.850773 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 10:52:47.851643 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 10:52:47.851643 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application with Lock( Aug 26 10:52:47.851643 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 10:52:47.851643 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._enter() Aug 26 10:52:47.851643 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 10:52:47.851643 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 10:52:47.851643 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 10:52:47.851643 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.creator() Aug 26 10:52:47.851643 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 10:52:47.851643 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application created_value = creator( Aug 26 10:52:47.851643 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 26 10:52:47.851643 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 26 10:52:47.851643 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 26 10:52:47.851643 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Aug 26 10:52:47.858147 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 445/892] 10.222.0.40 () {56 vars in 1114 bytes} [Fri Aug 26 10:52:47 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 26 10:52:47.865291 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0611ad3a-502a-46ec-8f16-9e4f49de3027 None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:52:47.871213 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084097cb-d566-4e1c-885f-15752391694f None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:52:47.905814 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:52:47.908066 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7baa8296-2786-4a96-bff4-57bb4181c08e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:52:47.908679 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7baa8296-2786-4a96-bff4-57bb4181c08e None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:47.909030 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7baa8296-2786-4a96-bff4-57bb4181c08e None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:47.909301 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7baa8296-2786-4a96-bff4-57bb4181c08e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:47.929045 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:52:47.979201 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7baa8296-2786-4a96-bff4-57bb4181c08e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:52:47.980723 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7baa8296-2786-4a96-bff4-57bb4181c08e None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:52:47.982066 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0611ad3a-502a-46ec-8f16-9e4f49de3027 None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:52:48.019521 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f2d316-8eaf-48fe-9471-123128f78a8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:52:48.020813 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f2d316-8eaf-48fe-9471-123128f78a8b None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:48.021240 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f2d316-8eaf-48fe-9471-123128f78a8b None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:48.021746 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f2d316-8eaf-48fe-9471-123128f78a8b None admin] PATH_INFO: `/v3/services` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:48.022793 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f2d316-8eaf-48fe-9471-123128f78a8b None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 10:52:48.023320 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f2d316-8eaf-48fe-9471-123128f78a8b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:52:48.025028 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f2d316-8eaf-48fe-9471-123128f78a8b None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:52:48.030980 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 446/893] 10.222.0.40 () {56 vars in 1114 bytes} [Fri Aug 26 10:52:47 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 10:52:48.042051 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f2d316-8eaf-48fe-9471-123128f78a8b None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:52:48.087799 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 448/894] 10.222.0.40 () {58 vars in 1282 bytes} [Fri Aug 26 10:52:47 2022] GET /identity/v3/auth/tokens => generated 1372 bytes in 358 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 26 10:52:48.093031 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7baa8296-2786-4a96-bff4-57bb4181c08e None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:52:48.136715 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:52:48.158313 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:52:48.166509 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f2d316-8eaf-48fe-9471-123128f78a8b None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:52:48.199678 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7baa8296-2786-4a96-bff4-57bb4181c08e None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:52:48.205280 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237ce30a-9882-422f-ab8b-70a72cfa4bcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:52:48.205891 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237ce30a-9882-422f-ab8b-70a72cfa4bcd None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:48.206193 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237ce30a-9882-422f-ab8b-70a72cfa4bcd None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:48.206469 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237ce30a-9882-422f-ab8b-70a72cfa4bcd None admin] PATH_INFO: `/v3/services` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:48.207098 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-237ce30a-9882-422f-ab8b-70a72cfa4bcd None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 10:52:48.207396 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-237ce30a-9882-422f-ab8b-70a72cfa4bcd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:52:48.208751 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-237ce30a-9882-422f-ab8b-70a72cfa4bcd None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:52:48.214580 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 447/895] 10.222.0.40 () {56 vars in 1114 bytes} [Fri Aug 26 10:52:48 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 10:52:48.216654 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bf453c-cede-43d4-a6d4-0e878a532185 None None] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:48.217154 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bf453c-cede-43d4-a6d4-0e878a532185 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:48.217792 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bf453c-cede-43d4-a6d4-0e878a532185 None None] PATH_INFO: `/` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:48.219840 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 448/896] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 26 10:52:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 10:52:48.220792 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0bf453c-cede-43d4-a6d4-0e878a532185 None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:52:48.235917 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acbbb1f7-4cd8-4ce5-8ab3-8d7fe4129225 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:52:48.236476 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbbb1f7-4cd8-4ce5-8ab3-8d7fe4129225 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:48.236754 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbbb1f7-4cd8-4ce5-8ab3-8d7fe4129225 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:48.237023 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbbb1f7-4cd8-4ce5-8ab3-8d7fe4129225 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:48.247166 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acbbb1f7-4cd8-4ce5-8ab3-8d7fe4129225 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:52:48.248608 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acbbb1f7-4cd8-4ce5-8ab3-8d7fe4129225 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:52:48.263709 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application [None req-acbbb1f7-4cd8-4ce5-8ab3-8d7fe4129225 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 10:52:48.263709 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 10:52:48.263709 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 10:52:48.263709 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 10:52:48.263709 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 10:52:48.263709 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 10:52:48.263709 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 10:52:48.263709 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 10:52:48.263709 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 10:52:48.263709 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 10:52:48.263709 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 10:52:48.263709 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 10:52:48.263709 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 10:52:48.263709 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 10:52:48.263709 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 10:52:48.263709 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 10:52:48.264690 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 10:52:48.264690 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 10:52:48.264690 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 10:52:48.264690 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 10:52:48.264690 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 10:52:48.264690 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 10:52:48.264690 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 10:52:48.264690 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application with Lock( Aug 26 10:52:48.264690 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 10:52:48.264690 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._enter() Aug 26 10:52:48.264690 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 10:52:48.264690 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 10:52:48.264690 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 10:52:48.264690 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.creator() Aug 26 10:52:48.264690 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 10:52:48.264690 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application created_value = creator( Aug 26 10:52:48.264690 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 10:52:48.265650 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 10:52:48.265650 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 10:52:48.265650 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Aug 26 10:52:48.265650 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 449/897] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Aug 26 10:52:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 26 10:52:48.266916 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af5c07c-c323-460f-9f40-119660f1abeb None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:48.266916 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af5c07c-c323-460f-9f40-119660f1abeb None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:48.267089 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af5c07c-c323-460f-9f40-119660f1abeb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:48.284602 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-4af5c07c-c323-460f-9f40-119660f1abeb None None] MFA Rules not processed for user `ca155a48e2024c32af3fd2e19e5fac11`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 10:52:48.285794 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-4af5c07c-c323-460f-9f40-119660f1abeb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 10:52:48.350243 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0bf453c-cede-43d4-a6d4-0e878a532185 None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:52:48.356563 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 450/898] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 26 10:52:48 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 26 10:52:48.357239 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af5c07c-c323-460f-9f40-119660f1abeb None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:52:48.381876 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:52:48.399016 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259bdc02-fe4b-47c4-ae18-120d0fbbb90d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:52:48.399890 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259bdc02-fe4b-47c4-ae18-120d0fbbb90d None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:48.400163 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259bdc02-fe4b-47c4-ae18-120d0fbbb90d None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:48.400426 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259bdc02-fe4b-47c4-ae18-120d0fbbb90d None admin] PATH_INFO: `/v3/endpoints` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:48.401135 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-259bdc02-fe4b-47c4-ae18-120d0fbbb90d None admin] RBAC: Adding query filter params (interface=public, service_id=b3e5bf9ea9714652a4910b50152bd5c8, region_id=RegionOne) {{(pid=81423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 10:52:48.401441 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-259bdc02-fe4b-47c4-ae18-120d0fbbb90d None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:52:48.403689 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-259bdc02-fe4b-47c4-ae18-120d0fbbb90d None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:52:48.413932 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 449/899] 10.222.0.40 () {56 vars in 1251 bytes} [Fri Aug 26 10:52:48 2022] GET /identity/v3/endpoints?service_id=b3e5bf9ea9714652a4910b50152bd5c8&interface=public®ion_id=RegionOne => generated 194 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 10:52:48.413932 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d4e8cd-992a-47dc-9023-032b5390b90e None None] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:48.413932 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d4e8cd-992a-47dc-9023-032b5390b90e None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:48.413932 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d4e8cd-992a-47dc-9023-032b5390b90e None None] PATH_INFO: `/` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:48.413932 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 450/900] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 26 10:52:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 10:52:48.417811 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d4e8cd-992a-47dc-9023-032b5390b90e None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:52:48.445063 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:52:48.473365 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af5c07c-c323-460f-9f40-119660f1abeb None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:52:48.510030 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eaf84a0-1859-4673-832c-0871a15f329e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:52:48.510567 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eaf84a0-1859-4673-832c-0871a15f329e None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:48.510850 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eaf84a0-1859-4673-832c-0871a15f329e None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:48.511127 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eaf84a0-1859-4673-832c-0871a15f329e None admin] PATH_INFO: `/v3/domains` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:48.511768 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eaf84a0-1859-4673-832c-0871a15f329e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 10:52:48.512071 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eaf84a0-1859-4673-832c-0871a15f329e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:52:48.513437 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eaf84a0-1859-4673-832c-0871a15f329e None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:52:48.529859 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 451/901] 10.222.0.40 () {56 vars in 1109 bytes} [Fri Aug 26 10:52:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 10:52:48.529859 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f211d4-029b-4261-8835-b1b44843636b None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:48.529859 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f211d4-029b-4261-8835-b1b44843636b None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:48.529859 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f211d4-029b-4261-8835-b1b44843636b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:48.531902 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d4e8cd-992a-47dc-9023-032b5390b90e None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:52:48.545888 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-a7f211d4-029b-4261-8835-b1b44843636b None None] MFA Rules not processed for user `ca155a48e2024c32af3fd2e19e5fac11`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 10:52:48.547212 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-a7f211d4-029b-4261-8835-b1b44843636b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 10:52:48.569592 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8928fb6-61f0-4447-a515-b7cb27d1a62c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:52:48.570310 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8928fb6-61f0-4447-a515-b7cb27d1a62c None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:48.570653 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8928fb6-61f0-4447-a515-b7cb27d1a62c None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:48.570987 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8928fb6-61f0-4447-a515-b7cb27d1a62c None admin] PATH_INFO: `/v3/services` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:48.571652 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8928fb6-61f0-4447-a515-b7cb27d1a62c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:52:48.573056 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8928fb6-61f0-4447-a515-b7cb27d1a62c None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:52:48.579213 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 451/902] 10.222.0.40 () {56 vars in 1087 bytes} [Fri Aug 26 10:52:48 2022] GET /identity/v3/services => generated 1268 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 10:52:48.579942 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8928fb6-61f0-4447-a515-b7cb27d1a62c None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:52:48.646415 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 452/903] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 26 10:52:48 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 26 10:52:48.647174 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f211d4-029b-4261-8835-b1b44843636b None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:52:48.664822 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:52:48.695182 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:52:48.713179 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8928fb6-61f0-4447-a515-b7cb27d1a62c None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:52:48.757996 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca887137-fdf1-4b6b-a486-e8703aadc834 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:52:48.758639 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca887137-fdf1-4b6b-a486-e8703aadc834 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:48.758920 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca887137-fdf1-4b6b-a486-e8703aadc834 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:48.759193 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca887137-fdf1-4b6b-a486-e8703aadc834 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:48.767453 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca887137-fdf1-4b6b-a486-e8703aadc834 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:52:48.768832 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca887137-fdf1-4b6b-a486-e8703aadc834 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:52:48.779897 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f211d4-029b-4261-8835-b1b44843636b None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:52:48.786863 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application [None req-ca887137-fdf1-4b6b-a486-e8703aadc834 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 10:52:48.786863 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 10:52:48.786863 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 10:52:48.786863 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 10:52:48.786863 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 10:52:48.786863 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 10:52:48.786863 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 10:52:48.786863 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 10:52:48.786863 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 10:52:48.786863 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 10:52:48.786863 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 10:52:48.786863 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 10:52:48.786863 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 26 10:52:48.786863 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 26 10:52:48.786863 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 26 10:52:48.787864 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 26 10:52:48.787864 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 10:52:48.787864 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 10:52:48.787864 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 10:52:48.787864 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 10:52:48.787864 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 10:52:48.787864 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 10:52:48.787864 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 10:52:48.787864 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application with Lock( Aug 26 10:52:48.787864 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 10:52:48.787864 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._enter() Aug 26 10:52:48.787864 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 10:52:48.787864 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 10:52:48.787864 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 10:52:48.787864 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.creator() Aug 26 10:52:48.787864 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 10:52:48.787864 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application created_value = creator( Aug 26 10:52:48.788747 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 26 10:52:48.788747 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 26 10:52:48.788747 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 26 10:52:48.788747 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 26 10:52:48.788747 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 26 10:52:48.788747 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 26 10:52:48.788747 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 10:52:48.788747 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Aug 26 10:52:48.792741 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 452/904] 10.222.0.40 () {56 vars in 1111 bytes} [Fri Aug 26 10:52:48 2022] GET /identity/v3/projects/service => generated 88 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 10:52:48.793954 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca887137-fdf1-4b6b-a486-e8703aadc834 None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:52:48.824057 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6eaaac-4f84-476f-ab12-96301ba0071b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:52:48.824975 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6eaaac-4f84-476f-ab12-96301ba0071b None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:48.825397 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6eaaac-4f84-476f-ab12-96301ba0071b None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:48.825811 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6eaaac-4f84-476f-ab12-96301ba0071b None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:48.826590 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a6eaaac-4f84-476f-ab12-96301ba0071b None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:52:48.828134 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a6eaaac-4f84-476f-ab12-96301ba0071b None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:52:48.837787 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application [None req-7a6eaaac-4f84-476f-ab12-96301ba0071b None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 26 10:52:48.837787 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 10:52:48.837787 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 26 10:52:48.837787 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 26 10:52:48.837787 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 26 10:52:48.837787 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 26 10:52:48.837787 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 26 10:52:48.837787 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 26 10:52:48.837787 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 26 10:52:48.837787 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Aug 26 10:52:48.837787 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 26 10:52:48.837787 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Aug 26 10:52:48.837787 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 10:52:48.837787 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 10:52:48.837787 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 10:52:48.837787 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 10:52:48.838979 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 10:52:48.838979 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 10:52:48.838979 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 10:52:48.838979 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 10:52:48.838979 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 10:52:48.838979 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 10:52:48.838979 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 10:52:48.838979 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 26 10:52:48.838979 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 26 10:52:48.838979 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 26 10:52:48.838979 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 26 10:52:48.838979 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 10:52:48.838979 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 10:52:48.838979 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 10:52:48.838979 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 10:52:48.838979 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 10:52:48.838979 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 10:52:48.840107 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 10:52:48.840107 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application with Lock( Aug 26 10:52:48.840107 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 10:52:48.840107 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._enter() Aug 26 10:52:48.840107 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 10:52:48.840107 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 10:52:48.840107 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 10:52:48.840107 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.creator() Aug 26 10:52:48.840107 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 10:52:48.840107 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application created_value = creator( Aug 26 10:52:48.840107 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 26 10:52:48.840107 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 26 10:52:48.840107 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 26 10:52:48.840107 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Aug 26 10:52:48.842761 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 453/905] 10.222.0.40 () {56 vars in 1132 bytes} [Fri Aug 26 10:52:48 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 10:52:48.849469 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6eaaac-4f84-476f-ab12-96301ba0071b None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:52:48.916161 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca887137-fdf1-4b6b-a486-e8703aadc834 None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:52:48.951682 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a787a4-c703-4775-8257-fec39f5ccc67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:52:48.952284 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a787a4-c703-4775-8257-fec39f5ccc67 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:48.952554 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a787a4-c703-4775-8257-fec39f5ccc67 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:48.952818 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a787a4-c703-4775-8257-fec39f5ccc67 None admin] PATH_INFO: `/v3/projects` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:48.953542 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a787a4-c703-4775-8257-fec39f5ccc67 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 10:52:48.953846 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a787a4-c703-4775-8257-fec39f5ccc67 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:52:48.955300 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a787a4-c703-4775-8257-fec39f5ccc67 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:52:48.955631 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:52:48.973147 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 453/906] 10.222.0.40 () {56 vars in 1148 bytes} [Fri Aug 26 10:52:48 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 10:52:48.973147 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a787a4-c703-4775-8257-fec39f5ccc67 None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:52:48.984116 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:52:48.989535 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6eaaac-4f84-476f-ab12-96301ba0071b None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:52:49.027856 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a88644-d992-49f8-b638-fc16c7f42279 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:52:49.028629 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a88644-d992-49f8-b638-fc16c7f42279 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:49.028742 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a88644-d992-49f8-b638-fc16c7f42279 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:49.028925 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a88644-d992-49f8-b638-fc16c7f42279 None admin] PATH_INFO: `/v3/services` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:49.029576 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a88644-d992-49f8-b638-fc16c7f42279 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 10:52:49.029858 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a88644-d992-49f8-b638-fc16c7f42279 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:52:49.031279 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a88644-d992-49f8-b638-fc16c7f42279 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:52:49.041668 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 454/907] 10.222.0.40 () {56 vars in 1126 bytes} [Fri Aug 26 10:52:48 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 10:52:49.041668 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a88644-d992-49f8-b638-fc16c7f42279 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:52:49.083117 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a787a4-c703-4775-8257-fec39f5ccc67 None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:52:49.121235 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b699bc99-9f62-4371-b4e3-e407b5a2ea13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:52:49.121838 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b699bc99-9f62-4371-b4e3-e407b5a2ea13 None admin] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:49.122136 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b699bc99-9f62-4371-b4e3-e407b5a2ea13 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:49.122408 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b699bc99-9f62-4371-b4e3-e407b5a2ea13 None admin] PATH_INFO: `/v3/services` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:49.123047 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b699bc99-9f62-4371-b4e3-e407b5a2ea13 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:52:49.124468 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b699bc99-9f62-4371-b4e3-e407b5a2ea13 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:52:49.144273 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 454/908] 10.222.0.40 () {60 vars in 1141 bytes} [Fri Aug 26 10:52:48 2022] POST /identity/v3/services => generated 241 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 10:52:49.145211 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b699bc99-9f62-4371-b4e3-e407b5a2ea13 None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:52:49.160122 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a88644-d992-49f8-b638-fc16c7f42279 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:52:49.210599 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb73452-a793-45f2-9eec-356e81a6d8d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:52:49.211223 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb73452-a793-45f2-9eec-356e81a6d8d9 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:49.211497 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb73452-a793-45f2-9eec-356e81a6d8d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:49.211766 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb73452-a793-45f2-9eec-356e81a6d8d9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:49.212498 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb73452-a793-45f2-9eec-356e81a6d8d9 None admin] RBAC: Adding query filter params (role.id=3f7d20781437466aabcbf11bda248f3d, scope.project.id=0e30ccf0c1204727b786876c141dbcd2, user.id=50be12ccbd274a8792615e9dd10b61ca) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 10:52:49.212896 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb73452-a793-45f2-9eec-356e81a6d8d9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:52:49.214408 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb73452-a793-45f2-9eec-356e81a6d8d9 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:52:49.217807 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:52:49.223716 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 455/909] 10.222.0.40 () {56 vars in 1374 bytes} [Fri Aug 26 10:52:49 2022] GET /identity/v3/role_assignments?user.id=50be12ccbd274a8792615e9dd10b61ca&scope.project.id=0e30ccf0c1204727b786876c141dbcd2&role.id=3f7d20781437466aabcbf11bda248f3d => generated 259 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 10:52:49.264148 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:52:49.296347 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b699bc99-9f62-4371-b4e3-e407b5a2ea13 None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:52:49.336090 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea03b8e-fa18-420d-b31e-0eaf10ff405a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:52:49.336775 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea03b8e-fa18-420d-b31e-0eaf10ff405a None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:49.337051 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea03b8e-fa18-420d-b31e-0eaf10ff405a None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:49.337320 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea03b8e-fa18-420d-b31e-0eaf10ff405a None admin] PATH_INFO: `/v3/services` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:49.338040 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea03b8e-fa18-420d-b31e-0eaf10ff405a None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 10:52:49.338346 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea03b8e-fa18-420d-b31e-0eaf10ff405a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:52:49.339726 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea03b8e-fa18-420d-b31e-0eaf10ff405a None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:52:49.346396 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 455/910] 10.222.0.40 () {56 vars in 1126 bytes} [Fri Aug 26 10:52:49 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 10:52:49.352683 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb73452-a793-45f2-9eec-356e81a6d8d9 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:52:49.466182 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb73452-a793-45f2-9eec-356e81a6d8d9 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:52:49.502174 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56eb747-12f9-47a5-b357-d526198b6ea4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:52:49.503170 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56eb747-12f9-47a5-b357-d526198b6ea4 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:49.503663 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56eb747-12f9-47a5-b357-d526198b6ea4 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:49.504153 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56eb747-12f9-47a5-b357-d526198b6ea4 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:49.505068 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a56eb747-12f9-47a5-b357-d526198b6ea4 None admin] RBAC: Adding query filter params (interface=public, service_id=3e434ae377f34a789e35e49a2e709bd2, region_id=RegionOne) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 10:52:49.505597 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a56eb747-12f9-47a5-b357-d526198b6ea4 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:52:49.507231 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a56eb747-12f9-47a5-b357-d526198b6ea4 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:52:49.520686 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 456/911] 10.222.0.40 () {56 vars in 1251 bytes} [Fri Aug 26 10:52:49 2022] GET /identity/v3/endpoints?service_id=3e434ae377f34a789e35e49a2e709bd2&interface=public®ion_id=RegionOne => generated 194 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 10:52:49.522340 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea03b8e-fa18-420d-b31e-0eaf10ff405a None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:52:49.627862 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea03b8e-fa18-420d-b31e-0eaf10ff405a None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:52:49.658563 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:52:49.663761 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80cb7903-0246-4331-b7e0-1cc38ef51501 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:52:49.664373 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80cb7903-0246-4331-b7e0-1cc38ef51501 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:49.664644 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80cb7903-0246-4331-b7e0-1cc38ef51501 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:49.664909 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80cb7903-0246-4331-b7e0-1cc38ef51501 None admin] PATH_INFO: `/v3/services` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:49.665727 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80cb7903-0246-4331-b7e0-1cc38ef51501 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:52:49.667135 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80cb7903-0246-4331-b7e0-1cc38ef51501 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:52:49.674009 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 456/912] 10.222.0.40 () {56 vars in 1087 bytes} [Fri Aug 26 10:52:49 2022] GET /identity/v3/services => generated 1497 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 10:52:50.588306 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e8939a-76eb-4475-9df5-661c93bf1e3e None None] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:50.588845 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e8939a-76eb-4475-9df5-661c93bf1e3e None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:50.588845 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e8939a-76eb-4475-9df5-661c93bf1e3e None None] PATH_INFO: `/` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:50.590738 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 457/913] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 26 10:52:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 10:52:50.596666 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f409e219-d772-480c-920f-d4012a53e972 None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:50.597065 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f409e219-d772-480c-920f-d4012a53e972 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:50.597228 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f409e219-d772-480c-920f-d4012a53e972 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:50.616470 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-f409e219-d772-480c-920f-d4012a53e972 None None] MFA Rules not processed for user `ca155a48e2024c32af3fd2e19e5fac11`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 10:52:50.617732 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-f409e219-d772-480c-920f-d4012a53e972 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 10:52:50.687306 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 457/914] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 26 10:52:50 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 26 10:52:50.964093 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e8939a-76eb-4475-9df5-661c93bf1e3e None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:52:51.033610 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:52:51.074788 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e8939a-76eb-4475-9df5-661c93bf1e3e None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:52:51.111070 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f398a34-57c7-4353-a74a-8b32e75e7807 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:52:51.111699 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f398a34-57c7-4353-a74a-8b32e75e7807 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:51.111973 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f398a34-57c7-4353-a74a-8b32e75e7807 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:51.112240 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f398a34-57c7-4353-a74a-8b32e75e7807 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:51.112819 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f398a34-57c7-4353-a74a-8b32e75e7807 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:52:51.114200 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f398a34-57c7-4353-a74a-8b32e75e7807 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:52:51.123060 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application [None req-1f398a34-57c7-4353-a74a-8b32e75e7807 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 26 10:52:51.123060 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 10:52:51.123060 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 26 10:52:51.123060 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 26 10:52:51.123060 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 26 10:52:51.123060 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 26 10:52:51.123060 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 26 10:52:51.123060 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 26 10:52:51.123060 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 26 10:52:51.123060 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Aug 26 10:52:51.123060 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 26 10:52:51.123060 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Aug 26 10:52:51.123060 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 10:52:51.123060 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 10:52:51.123060 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 10:52:51.123060 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 10:52:51.123060 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 10:52:51.125969 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 10:52:51.125969 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 10:52:51.125969 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 10:52:51.125969 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 10:52:51.125969 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 10:52:51.125969 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 10:52:51.125969 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 26 10:52:51.125969 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 26 10:52:51.125969 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 26 10:52:51.125969 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 26 10:52:51.125969 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 10:52:51.125969 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 10:52:51.125969 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 10:52:51.125969 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 10:52:51.125969 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 10:52:51.125969 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 10:52:51.126934 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 10:52:51.126934 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application with Lock( Aug 26 10:52:51.126934 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 10:52:51.126934 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._enter() Aug 26 10:52:51.126934 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 10:52:51.126934 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 10:52:51.126934 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 10:52:51.126934 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.creator() Aug 26 10:52:51.126934 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 10:52:51.126934 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application created_value = creator( Aug 26 10:52:51.126934 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 26 10:52:51.126934 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 26 10:52:51.126934 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 26 10:52:51.126934 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Aug 26 10:52:51.126934 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 458/915] 10.222.0.40 () {56 vars in 1114 bytes} [Fri Aug 26 10:52:50 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 10:52:51.131159 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f409e219-d772-480c-920f-d4012a53e972 None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:52:51.218994 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14d3832-cf20-4442-a309-bf538a5d3763 None None] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:51.219861 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14d3832-cf20-4442-a309-bf538a5d3763 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:51.220038 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14d3832-cf20-4442-a309-bf538a5d3763 None None] PATH_INFO: `/` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:51.221494 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 459/916] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 26 10:52:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 10:52:51.227379 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ed4534-11c3-4894-b824-68060594f1a3 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:51.227741 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ed4534-11c3-4894-b824-68060594f1a3 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:51.228017 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ed4534-11c3-4894-b824-68060594f1a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:51.240030 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f409e219-d772-480c-920f-d4012a53e972 None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:52:51.247135 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-b6ed4534-11c3-4894-b824-68060594f1a3 None None] MFA Rules not processed for user `ca155a48e2024c32af3fd2e19e5fac11`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 10:52:51.248326 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-b6ed4534-11c3-4894-b824-68060594f1a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 10:52:51.273421 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:52:51.283241 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40644fb8-b6cc-4fa0-9dbc-ad0fecf74405 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:52:51.283869 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40644fb8-b6cc-4fa0-9dbc-ad0fecf74405 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:51.284142 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40644fb8-b6cc-4fa0-9dbc-ad0fecf74405 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:51.284417 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40644fb8-b6cc-4fa0-9dbc-ad0fecf74405 None admin] PATH_INFO: `/v3/services` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:51.285045 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40644fb8-b6cc-4fa0-9dbc-ad0fecf74405 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 10:52:51.285507 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40644fb8-b6cc-4fa0-9dbc-ad0fecf74405 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:52:51.287699 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40644fb8-b6cc-4fa0-9dbc-ad0fecf74405 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:52:51.293533 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 458/917] 10.222.0.40 () {56 vars in 1114 bytes} [Fri Aug 26 10:52:51 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 10:52:51.300428 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40644fb8-b6cc-4fa0-9dbc-ad0fecf74405 None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:52:51.327383 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 460/918] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 26 10:52:51 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 26 10:52:51.328429 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3c1a68-530b-4adc-9779-79277797212a None None] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:51.328761 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3c1a68-530b-4adc-9779-79277797212a None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:51.329063 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3c1a68-530b-4adc-9779-79277797212a None None] PATH_INFO: `/` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:51.330340 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 461/919] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 26 10:52:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 10:52:51.338342 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0375867a-1168-4a9a-a959-5cecdaa13da7 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:51.338757 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0375867a-1168-4a9a-a959-5cecdaa13da7 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:51.339130 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0375867a-1168-4a9a-a959-5cecdaa13da7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:51.359637 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-0375867a-1168-4a9a-a959-5cecdaa13da7 None None] MFA Rules not processed for user `ca155a48e2024c32af3fd2e19e5fac11`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 10:52:51.361698 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-0375867a-1168-4a9a-a959-5cecdaa13da7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 10:52:51.425890 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40644fb8-b6cc-4fa0-9dbc-ad0fecf74405 None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:52:51.429902 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:52:51.454133 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 462/920] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 26 10:52:51 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 26 10:52:51.468650 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265bc419-dbfb-4e14-8f11-2382c0fd84fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:52:51.470265 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265bc419-dbfb-4e14-8f11-2382c0fd84fe None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:51.470812 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265bc419-dbfb-4e14-8f11-2382c0fd84fe None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:51.471295 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265bc419-dbfb-4e14-8f11-2382c0fd84fe None admin] PATH_INFO: `/v3/services` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:51.472231 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-265bc419-dbfb-4e14-8f11-2382c0fd84fe None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 10:52:51.472731 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-265bc419-dbfb-4e14-8f11-2382c0fd84fe None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:52:51.474410 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-265bc419-dbfb-4e14-8f11-2382c0fd84fe None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:52:51.479882 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 459/921] 10.222.0.40 () {56 vars in 1114 bytes} [Fri Aug 26 10:52:51 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 10:52:51.486055 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0375867a-1168-4a9a-a959-5cecdaa13da7 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:52:51.586476 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0375867a-1168-4a9a-a959-5cecdaa13da7 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:52:51.611163 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265bc419-dbfb-4e14-8f11-2382c0fd84fe None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:52:51.635742 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6cfda9-b331-4024-9567-935ede739d4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:52:51.636412 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6cfda9-b331-4024-9567-935ede739d4f None admin] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:51.636636 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6cfda9-b331-4024-9567-935ede739d4f None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:51.636938 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6cfda9-b331-4024-9567-935ede739d4f None admin] PATH_INFO: `/v3/endpoints` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:51.637874 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e6cfda9-b331-4024-9567-935ede739d4f None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:52:51.640050 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e6cfda9-b331-4024-9567-935ede739d4f None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:52:51.667542 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 463/922] 10.222.0.40 () {60 vars in 1144 bytes} [Fri Aug 26 10:52:51 2022] POST /identity/v3/endpoints => generated 346 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 10:52:51.668849 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ba9f21-d03d-496e-a368-f50f215e4f93 None None] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:51.669139 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ba9f21-d03d-496e-a368-f50f215e4f93 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:51.669410 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ba9f21-d03d-496e-a368-f50f215e4f93 None None] PATH_INFO: `/` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:51.671564 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 464/923] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 26 10:52:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 10:52:51.680059 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f4acb6-fdca-4e54-9373-cd65c9420e55 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:51.680238 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f4acb6-fdca-4e54-9373-cd65c9420e55 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:51.680522 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f4acb6-fdca-4e54-9373-cd65c9420e55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:51.696290 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:52:51.703171 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-30f4acb6-fdca-4e54-9373-cd65c9420e55 None None] MFA Rules not processed for user `ca155a48e2024c32af3fd2e19e5fac11`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 10:52:51.705641 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-30f4acb6-fdca-4e54-9373-cd65c9420e55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 10:52:51.759761 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265bc419-dbfb-4e14-8f11-2382c0fd84fe None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:52:51.762952 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:52:51.804978 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16607656-804e-41f0-8314-e9ddb6feca8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:52:51.810026 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 465/924] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 26 10:52:51 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 26 10:52:51.810253 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16607656-804e-41f0-8314-e9ddb6feca8f None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:51.810253 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16607656-804e-41f0-8314-e9ddb6feca8f None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:51.810253 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16607656-804e-41f0-8314-e9ddb6feca8f None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:51.810484 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f4acb6-fdca-4e54-9373-cd65c9420e55 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:52:51.810556 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16607656-804e-41f0-8314-e9ddb6feca8f None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:52:51.810556 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16607656-804e-41f0-8314-e9ddb6feca8f None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:52:51.822060 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application [None req-16607656-804e-41f0-8314-e9ddb6feca8f None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 26 10:52:51.822060 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 10:52:51.822060 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 26 10:52:51.822060 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 26 10:52:51.822060 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 26 10:52:51.822060 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 26 10:52:51.822060 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 26 10:52:51.822060 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 26 10:52:51.822060 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 26 10:52:51.822060 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Aug 26 10:52:51.822060 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 26 10:52:51.822060 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Aug 26 10:52:51.822060 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 10:52:51.822060 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 10:52:51.822060 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 10:52:51.822060 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 10:52:51.822060 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 10:52:51.823377 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 10:52:51.823377 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 10:52:51.823377 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 10:52:51.823377 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 10:52:51.823377 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 10:52:51.823377 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 10:52:51.823377 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 26 10:52:51.823377 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 26 10:52:51.823377 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 26 10:52:51.823377 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 26 10:52:51.823377 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 10:52:51.823377 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 10:52:51.823377 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 10:52:51.823377 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 10:52:51.823377 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 10:52:51.823377 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 10:52:51.824386 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 10:52:51.824386 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application with Lock( Aug 26 10:52:51.824386 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 10:52:51.824386 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._enter() Aug 26 10:52:51.824386 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 10:52:51.824386 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 10:52:51.824386 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 10:52:51.824386 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.creator() Aug 26 10:52:51.824386 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 10:52:51.824386 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application created_value = creator( Aug 26 10:52:51.824386 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 26 10:52:51.824386 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 26 10:52:51.824386 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 26 10:52:51.824386 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Aug 26 10:52:51.824386 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 460/925] 10.222.0.40 () {56 vars in 1126 bytes} [Fri Aug 26 10:52:51 2022] GET /identity/v3/services/object-store => generated 93 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 26 10:52:51.830853 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16607656-804e-41f0-8314-e9ddb6feca8f None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:52:51.936465 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f4acb6-fdca-4e54-9373-cd65c9420e55 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:52:51.973862 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16607656-804e-41f0-8314-e9ddb6feca8f None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:52:51.984212 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65afdb70-1eea-4484-8d20-94ef68e57252 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:52:51.984935 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65afdb70-1eea-4484-8d20-94ef68e57252 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:51.985220 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65afdb70-1eea-4484-8d20-94ef68e57252 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:51.985876 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65afdb70-1eea-4484-8d20-94ef68e57252 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:51.991161 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:52:52.000286 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65afdb70-1eea-4484-8d20-94ef68e57252 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 10:52:52.006997 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application [None req-65afdb70-1eea-4484-8d20-94ef68e57252 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 26 10:52:52.006997 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 10:52:52.006997 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 10:52:52.006997 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 10:52:52.006997 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 10:52:52.006997 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 10:52:52.006997 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 10:52:52.006997 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 10:52:52.006997 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 10:52:52.006997 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 10:52:52.006997 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 10:52:52.006997 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 10:52:52.006997 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 10:52:52.006997 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 10:52:52.006997 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 10:52:52.006997 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 10:52:52.007947 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 10:52:52.007947 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 10:52:52.007947 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 10:52:52.007947 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 10:52:52.007947 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 10:52:52.007947 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 10:52:52.007947 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 10:52:52.007947 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 10:52:52.007947 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 10:52:52.007947 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 10:52:52.007947 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 10:52:52.007947 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application with Lock( Aug 26 10:52:52.007947 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 10:52:52.007947 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._enter() Aug 26 10:52:52.007947 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 10:52:52.007947 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 10:52:52.008805 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 10:52:52.008805 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.creator() Aug 26 10:52:52.008805 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 10:52:52.008805 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application created_value = creator( Aug 26 10:52:52.008805 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 10:52:52.008805 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 10:52:52.008805 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 10:52:52.008805 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 10:52:52.008805 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 10:52:52.008805 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 10:52:52.008805 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 26 10:52:52.008805 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Aug 26 10:52:52.009469 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 466/926] 10.222.0.40 () {56 vars in 1120 bytes} [Fri Aug 26 10:52:51 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 10:52:52.014567 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65afdb70-1eea-4484-8d20-94ef68e57252 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:52:52.020172 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42067e1f-5bf2-4677-b266-8c08d32c953c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:52:52.020789 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42067e1f-5bf2-4677-b266-8c08d32c953c None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:52.021105 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42067e1f-5bf2-4677-b266-8c08d32c953c None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:52.021387 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42067e1f-5bf2-4677-b266-8c08d32c953c None admin] PATH_INFO: `/v3/services` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:52.022050 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42067e1f-5bf2-4677-b266-8c08d32c953c None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 10:52:52.022359 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42067e1f-5bf2-4677-b266-8c08d32c953c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:52:52.023781 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42067e1f-5bf2-4677-b266-8c08d32c953c None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:52:52.029501 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 461/927] 10.222.0.40 () {56 vars in 1122 bytes} [Fri Aug 26 10:52:51 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 10:52:52.035903 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42067e1f-5bf2-4677-b266-8c08d32c953c None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:52:52.046838 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:52:52.123961 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65afdb70-1eea-4484-8d20-94ef68e57252 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:52:52.144079 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42067e1f-5bf2-4677-b266-8c08d32c953c None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:52:52.160354 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6bfccc0-a60a-493a-b448-de6157806ec6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:52:52.161077 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bfccc0-a60a-493a-b448-de6157806ec6 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:52.161352 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bfccc0-a60a-493a-b448-de6157806ec6 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:52.161741 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bfccc0-a60a-493a-b448-de6157806ec6 None admin] PATH_INFO: `/v3/roles` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:52.162413 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6bfccc0-a60a-493a-b448-de6157806ec6 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 10:52:52.162713 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6bfccc0-a60a-493a-b448-de6157806ec6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:52:52.164076 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6bfccc0-a60a-493a-b448-de6157806ec6 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:52:52.178216 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 467/928] 10.222.0.40 () {56 vars in 1115 bytes} [Fri Aug 26 10:52:52 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 10:52:52.178216 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6bfccc0-a60a-493a-b448-de6157806ec6 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:52:52.183931 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7bb254-3539-4616-8d55-349eb37376f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:52:52.184547 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7bb254-3539-4616-8d55-349eb37376f8 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:52.184826 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7bb254-3539-4616-8d55-349eb37376f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:52.185155 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7bb254-3539-4616-8d55-349eb37376f8 None admin] PATH_INFO: `/v3/services` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:52.185791 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b7bb254-3539-4616-8d55-349eb37376f8 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 10:52:52.186133 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b7bb254-3539-4616-8d55-349eb37376f8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:52:52.187556 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b7bb254-3539-4616-8d55-349eb37376f8 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:52:52.193251 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 462/929] 10.222.0.40 () {56 vars in 1122 bytes} [Fri Aug 26 10:52:52 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 10:52:52.194077 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7bb254-3539-4616-8d55-349eb37376f8 None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:52:52.244793 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:52:52.280604 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:52:52.291172 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6bfccc0-a60a-493a-b448-de6157806ec6 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:52:52.305083 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7bb254-3539-4616-8d55-349eb37376f8 None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:52:52.327012 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ccc912b-85e7-4cbf-98aa-23e0808e5597 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:52:52.327642 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ccc912b-85e7-4cbf-98aa-23e0808e5597 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:52.328028 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ccc912b-85e7-4cbf-98aa-23e0808e5597 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:52.328296 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ccc912b-85e7-4cbf-98aa-23e0808e5597 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:52.329015 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ccc912b-85e7-4cbf-98aa-23e0808e5597 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:52:52.330404 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ccc912b-85e7-4cbf-98aa-23e0808e5597 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:52:52.339049 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a3ccb9-d3e9-4fb3-8a36-f4a97a8f9dc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:52:52.339284 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application [None req-3ccc912b-85e7-4cbf-98aa-23e0808e5597 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 26 10:52:52.339284 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 10:52:52.339284 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 26 10:52:52.339284 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 26 10:52:52.339284 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 26 10:52:52.339284 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 26 10:52:52.339284 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 26 10:52:52.339284 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 26 10:52:52.339284 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 26 10:52:52.339284 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Aug 26 10:52:52.339284 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 26 10:52:52.339284 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Aug 26 10:52:52.339284 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 10:52:52.339284 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 10:52:52.339284 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 10:52:52.339284 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 10:52:52.340219 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 10:52:52.340219 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 10:52:52.340219 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 10:52:52.340219 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 10:52:52.340219 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 10:52:52.340219 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 10:52:52.340219 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 10:52:52.340219 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 26 10:52:52.340219 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 26 10:52:52.340219 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 26 10:52:52.340219 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 26 10:52:52.340219 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 10:52:52.340219 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 10:52:52.340219 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 10:52:52.340219 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 10:52:52.340219 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 10:52:52.340219 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 10:52:52.343045 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 10:52:52.343045 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application with Lock( Aug 26 10:52:52.343045 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 10:52:52.343045 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._enter() Aug 26 10:52:52.343045 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 10:52:52.343045 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 10:52:52.343045 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 10:52:52.343045 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.creator() Aug 26 10:52:52.343045 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 10:52:52.343045 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application created_value = creator( Aug 26 10:52:52.343045 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 26 10:52:52.343045 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 26 10:52:52.343045 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 26 10:52:52.343045 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Aug 26 10:52:52.343795 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a3ccb9-d3e9-4fb3-8a36-f4a97a8f9dc1 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:52.343795 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a3ccb9-d3e9-4fb3-8a36-f4a97a8f9dc1 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:52.343795 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a3ccb9-d3e9-4fb3-8a36-f4a97a8f9dc1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:52.343979 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 468/930] 10.222.0.40 () {56 vars in 1132 bytes} [Fri Aug 26 10:52:52 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 10:52:52.343979 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ccc912b-85e7-4cbf-98aa-23e0808e5597 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:52:52.355888 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4a3ccb9-d3e9-4fb3-8a36-f4a97a8f9dc1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:52:52.357324 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4a3ccb9-d3e9-4fb3-8a36-f4a97a8f9dc1 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:52:52.372680 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application [None req-e4a3ccb9-d3e9-4fb3-8a36-f4a97a8f9dc1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 10:52:52.372680 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 10:52:52.372680 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 10:52:52.372680 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 10:52:52.372680 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 10:52:52.372680 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 10:52:52.372680 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 10:52:52.372680 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 10:52:52.372680 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 10:52:52.372680 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 10:52:52.372680 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 10:52:52.372680 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 10:52:52.372680 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 10:52:52.372680 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 10:52:52.372680 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 10:52:52.372680 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 10:52:52.373636 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 10:52:52.373636 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 10:52:52.373636 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 10:52:52.373636 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 10:52:52.373636 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 10:52:52.373636 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 10:52:52.373636 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 10:52:52.373636 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application with Lock( Aug 26 10:52:52.373636 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 10:52:52.373636 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._enter() Aug 26 10:52:52.373636 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 10:52:52.373636 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 10:52:52.373636 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 10:52:52.373636 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.creator() Aug 26 10:52:52.373636 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 10:52:52.373636 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application created_value = creator( Aug 26 10:52:52.373636 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 10:52:52.374546 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 10:52:52.374546 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 10:52:52.374546 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Aug 26 10:52:52.374546 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 463/931] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Aug 26 10:52:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 10:52:52.374546 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a3ccb9-d3e9-4fb3-8a36-f4a97a8f9dc1 None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:52:52.463035 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ccc912b-85e7-4cbf-98aa-23e0808e5597 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:52:52.479704 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:52:52.489326 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a3ccb9-d3e9-4fb3-8a36-f4a97a8f9dc1 None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:52:52.498941 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917859fe-1825-46a4-9aaf-c3dc39eec650 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:52:52.499551 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917859fe-1825-46a4-9aaf-c3dc39eec650 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:52.500183 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917859fe-1825-46a4-9aaf-c3dc39eec650 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:52.500995 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917859fe-1825-46a4-9aaf-c3dc39eec650 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:52.502030 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-917859fe-1825-46a4-9aaf-c3dc39eec650 None admin] RBAC: Adding query filter params (interface=public, service_id=1ad4e0d9c1544d42b1d4a571a6262d4e, region_id=RegionOne) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 10:52:52.502348 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-917859fe-1825-46a4-9aaf-c3dc39eec650 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:52:52.503914 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-917859fe-1825-46a4-9aaf-c3dc39eec650 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:52:52.513890 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 469/932] 10.222.0.40 () {56 vars in 1251 bytes} [Fri Aug 26 10:52:52 2022] GET /identity/v3/endpoints?service_id=1ad4e0d9c1544d42b1d4a571a6262d4e&interface=public®ion_id=RegionOne => generated 194 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 10:52:52.513890 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917859fe-1825-46a4-9aaf-c3dc39eec650 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:52:52.526031 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11101e0-f132-4023-8417-042299166bc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:52:52.526877 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11101e0-f132-4023-8417-042299166bc8 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:52.526877 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11101e0-f132-4023-8417-042299166bc8 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:52.527142 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11101e0-f132-4023-8417-042299166bc8 None admin] PATH_INFO: `/v3/services` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:52.527762 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d11101e0-f132-4023-8417-042299166bc8 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 10:52:52.528111 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d11101e0-f132-4023-8417-042299166bc8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:52:52.529452 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d11101e0-f132-4023-8417-042299166bc8 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:52:52.534798 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 464/933] 10.222.0.40 () {56 vars in 1126 bytes} [Fri Aug 26 10:52:52 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 10:52:52.535547 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11101e0-f132-4023-8417-042299166bc8 None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:52:52.542931 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:52:52.626973 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917859fe-1825-46a4-9aaf-c3dc39eec650 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:52:52.644445 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11101e0-f132-4023-8417-042299166bc8 None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:52:52.662870 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a08799-5af6-42ed-a998-2a736ced6c08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:52:52.663642 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a08799-5af6-42ed-a998-2a736ced6c08 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:52.663733 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a08799-5af6-42ed-a998-2a736ced6c08 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:52.664005 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a08799-5af6-42ed-a998-2a736ced6c08 None admin] PATH_INFO: `/v3/domains` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:52.664650 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a08799-5af6-42ed-a998-2a736ced6c08 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 10:52:52.664950 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a08799-5af6-42ed-a998-2a736ced6c08 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:52:52.666344 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a08799-5af6-42ed-a998-2a736ced6c08 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:52:52.678980 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 470/934] 10.222.0.40 () {56 vars in 1109 bytes} [Fri Aug 26 10:52:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 10:52:52.680102 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a08799-5af6-42ed-a998-2a736ced6c08 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:52:52.691171 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f20f19-e465-450b-b65f-dbe7338bcaa5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:52:52.691779 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f20f19-e465-450b-b65f-dbe7338bcaa5 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:52.692056 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f20f19-e465-450b-b65f-dbe7338bcaa5 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:52.692328 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f20f19-e465-450b-b65f-dbe7338bcaa5 None admin] PATH_INFO: `/v3/services` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:52.692933 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f20f19-e465-450b-b65f-dbe7338bcaa5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:52:52.694314 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f20f19-e465-450b-b65f-dbe7338bcaa5 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:52:52.700639 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 465/935] 10.222.0.40 () {56 vars in 1087 bytes} [Fri Aug 26 10:52:52 2022] GET /identity/v3/services => generated 1497 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 10:52:52.701408 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f20f19-e465-450b-b65f-dbe7338bcaa5 None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:52:52.734345 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:52:52.796872 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:52:52.825925 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a08799-5af6-42ed-a998-2a736ced6c08 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:52:52.843715 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f20f19-e465-450b-b65f-dbe7338bcaa5 None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:52:52.880103 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf6b3e6-0c80-403a-8e05-2cab67a9de64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:52:52.880743 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf6b3e6-0c80-403a-8e05-2cab67a9de64 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:52.881021 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf6b3e6-0c80-403a-8e05-2cab67a9de64 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:52.881717 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf6b3e6-0c80-403a-8e05-2cab67a9de64 None admin] PATH_INFO: `/v3/services` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:52.885706 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf6b3e6-0c80-403a-8e05-2cab67a9de64 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 10:52:52.885706 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf6b3e6-0c80-403a-8e05-2cab67a9de64 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:52:52.885706 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf6b3e6-0c80-403a-8e05-2cab67a9de64 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:52:52.892493 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 471/936] 10.222.0.40 () {56 vars in 1126 bytes} [Fri Aug 26 10:52:52 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 10:52:52.895325 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d414410b-9c24-4de1-bd22-630ba33b6f8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:52:52.895963 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d414410b-9c24-4de1-bd22-630ba33b6f8c None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:52.896240 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d414410b-9c24-4de1-bd22-630ba33b6f8c None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:52.896510 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d414410b-9c24-4de1-bd22-630ba33b6f8c None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:52.897938 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf6b3e6-0c80-403a-8e05-2cab67a9de64 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:52:52.915144 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d414410b-9c24-4de1-bd22-630ba33b6f8c None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:52:52.917685 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d414410b-9c24-4de1-bd22-630ba33b6f8c None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:52:52.934879 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application [None req-d414410b-9c24-4de1-bd22-630ba33b6f8c None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 26 10:52:52.934879 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 10:52:52.934879 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 10:52:52.934879 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 10:52:52.934879 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 10:52:52.934879 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 10:52:52.934879 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 10:52:52.934879 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 10:52:52.934879 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 10:52:52.934879 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 10:52:52.934879 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 10:52:52.934879 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 10:52:52.934879 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 26 10:52:52.934879 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 26 10:52:52.934879 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 26 10:52:52.934879 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 26 10:52:52.935864 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 10:52:52.935864 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 10:52:52.935864 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 26 10:52:52.935864 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 10:52:52.935864 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 26 10:52:52.935864 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 10:52:52.935864 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 10:52:52.935864 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 10:52:52.935864 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 10:52:52.935864 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 10:52:52.935864 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 10:52:52.935864 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application with Lock( Aug 26 10:52:52.935864 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 10:52:52.935864 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._enter() Aug 26 10:52:52.935864 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 10:52:52.935864 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 10:52:52.935864 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 10:52:52.935864 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.creator() Aug 26 10:52:52.936793 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 10:52:52.936793 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application created_value = creator( Aug 26 10:52:52.936793 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 26 10:52:52.936793 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 26 10:52:52.936793 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 26 10:52:52.936793 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 26 10:52:52.936793 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 26 10:52:52.936793 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 26 10:52:52.936793 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 26 10:52:52.936793 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Aug 26 10:52:52.940768 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 466/937] 10.222.0.40 () {56 vars in 1117 bytes} [Fri Aug 26 10:52:52 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 10:52:52.949895 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d414410b-9c24-4de1-bd22-630ba33b6f8c None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:52:53.015403 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf6b3e6-0c80-403a-8e05-2cab67a9de64 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:52:53.049913 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:52:53.055548 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a86d875-d2e1-4dc6-9a05-b5d9dc67c395 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:52:53.056274 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a86d875-d2e1-4dc6-9a05-b5d9dc67c395 None admin] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:53.056651 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a86d875-d2e1-4dc6-9a05-b5d9dc67c395 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:53.057017 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a86d875-d2e1-4dc6-9a05-b5d9dc67c395 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:53.057779 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a86d875-d2e1-4dc6-9a05-b5d9dc67c395 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:52:53.059265 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a86d875-d2e1-4dc6-9a05-b5d9dc67c395 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:52:53.073050 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d414410b-9c24-4de1-bd22-630ba33b6f8c None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:52:53.077662 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:52:53.084978 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 472/938] 10.222.0.40 () {60 vars in 1144 bytes} [Fri Aug 26 10:52:52 2022] POST /identity/v3/endpoints => generated 347 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 10:52:53.136614 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7759dff2-a60e-4c62-8107-561e5e8e274d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:52:53.137584 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7759dff2-a60e-4c62-8107-561e5e8e274d None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:53.138034 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7759dff2-a60e-4c62-8107-561e5e8e274d None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:53.138449 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7759dff2-a60e-4c62-8107-561e5e8e274d None admin] PATH_INFO: `/v3/users` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:53.139381 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7759dff2-a60e-4c62-8107-561e5e8e274d None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 10:52:53.139844 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7759dff2-a60e-4c62-8107-561e5e8e274d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:52:53.141642 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7759dff2-a60e-4c62-8107-561e5e8e274d None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:52:53.166943 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 467/939] 10.222.0.40 () {56 vars in 1149 bytes} [Fri Aug 26 10:52:52 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 10:52:53.173725 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a86d875-d2e1-4dc6-9a05-b5d9dc67c395 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:52:53.286997 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a86d875-d2e1-4dc6-9a05-b5d9dc67c395 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:52:53.323719 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196902d7-56c2-4b2b-a846-e39ecbc59a24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:52:53.324633 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196902d7-56c2-4b2b-a846-e39ecbc59a24 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:53.325058 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196902d7-56c2-4b2b-a846-e39ecbc59a24 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:53.325483 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196902d7-56c2-4b2b-a846-e39ecbc59a24 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:53.337322 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-196902d7-56c2-4b2b-a846-e39ecbc59a24 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:52:53.339026 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-196902d7-56c2-4b2b-a846-e39ecbc59a24 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:52:53.353677 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application [None req-196902d7-56c2-4b2b-a846-e39ecbc59a24 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 10:52:53.353677 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 10:52:53.353677 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 10:52:53.353677 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 10:52:53.353677 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 10:52:53.353677 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 10:52:53.353677 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 10:52:53.353677 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 10:52:53.353677 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 10:52:53.353677 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 10:52:53.353677 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 10:52:53.353677 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 10:52:53.353677 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 10:52:53.353677 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 10:52:53.353677 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 10:52:53.353677 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 10:52:53.354882 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 10:52:53.354882 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 10:52:53.354882 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 10:52:53.354882 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 10:52:53.354882 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 10:52:53.354882 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 10:52:53.354882 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 10:52:53.354882 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application with Lock( Aug 26 10:52:53.354882 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 10:52:53.354882 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._enter() Aug 26 10:52:53.354882 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 10:52:53.354882 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 10:52:53.354882 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 10:52:53.354882 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.creator() Aug 26 10:52:53.354882 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 10:52:53.354882 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application created_value = creator( Aug 26 10:52:53.354882 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 10:52:53.355962 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 10:52:53.355962 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 10:52:53.355962 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Aug 26 10:52:53.357698 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 473/940] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Aug 26 10:52:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 10:52:53.366033 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7759dff2-a60e-4c62-8107-561e5e8e274d None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:52:53.476987 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7759dff2-a60e-4c62-8107-561e5e8e274d None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:52:53.513378 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5a2a36-003a-48f2-8c5b-a2ddbec0102f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:52:53.513998 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5a2a36-003a-48f2-8c5b-a2ddbec0102f None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:53.514280 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5a2a36-003a-48f2-8c5b-a2ddbec0102f None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:53.514544 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5a2a36-003a-48f2-8c5b-a2ddbec0102f None admin] PATH_INFO: `/v3/domains` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:53.515171 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be5a2a36-003a-48f2-8c5b-a2ddbec0102f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 10:52:53.515462 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be5a2a36-003a-48f2-8c5b-a2ddbec0102f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:52:53.517556 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be5a2a36-003a-48f2-8c5b-a2ddbec0102f None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:52:53.519940 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:52:53.529712 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 468/941] 10.222.0.40 () {56 vars in 1109 bytes} [Fri Aug 26 10:52:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 10:52:53.535171 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196902d7-56c2-4b2b-a846-e39ecbc59a24 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:52:53.592181 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:52:53.643000 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196902d7-56c2-4b2b-a846-e39ecbc59a24 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:52:53.678714 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5bdc490-fa57-466c-ac52-e4c3f4537d1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:52:53.680152 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bdc490-fa57-466c-ac52-e4c3f4537d1a None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:53.680644 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bdc490-fa57-466c-ac52-e4c3f4537d1a None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:53.681113 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bdc490-fa57-466c-ac52-e4c3f4537d1a None admin] PATH_INFO: `/v3/projects/service` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:53.688856 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5bdc490-fa57-466c-ac52-e4c3f4537d1a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:52:53.690599 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5bdc490-fa57-466c-ac52-e4c3f4537d1a None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:52:53.702283 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application [None req-a5bdc490-fa57-466c-ac52-e4c3f4537d1a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 10:52:53.702283 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 10:52:53.702283 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 10:52:53.702283 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 10:52:53.702283 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 10:52:53.702283 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 10:52:53.702283 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 10:52:53.702283 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 10:52:53.702283 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 10:52:53.702283 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 10:52:53.702283 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 10:52:53.702283 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 10:52:53.702283 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 26 10:52:53.702283 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 26 10:52:53.702283 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 26 10:52:53.703186 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 26 10:52:53.703186 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 10:52:53.703186 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 10:52:53.703186 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 10:52:53.703186 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 10:52:53.703186 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 10:52:53.703186 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 10:52:53.703186 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 10:52:53.703186 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application with Lock( Aug 26 10:52:53.703186 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 10:52:53.703186 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._enter() Aug 26 10:52:53.703186 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 10:52:53.703186 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 10:52:53.703186 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 10:52:53.703186 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.creator() Aug 26 10:52:53.703186 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 10:52:53.703186 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application created_value = creator( Aug 26 10:52:53.704127 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 26 10:52:53.704127 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 26 10:52:53.704127 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 26 10:52:53.704127 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 26 10:52:53.704127 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 26 10:52:53.704127 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 26 10:52:53.704127 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 10:52:53.704127 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Aug 26 10:52:53.706731 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 474/942] 10.222.0.40 () {56 vars in 1111 bytes} [Fri Aug 26 10:52:53 2022] GET /identity/v3/projects/service => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 10:52:53.715282 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5a2a36-003a-48f2-8c5b-a2ddbec0102f None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:52:53.748724 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724b0144-c65c-4381-90fe-5c3d5c51ecdf None None] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:53.749236 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724b0144-c65c-4381-90fe-5c3d5c51ecdf None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:53.749656 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724b0144-c65c-4381-90fe-5c3d5c51ecdf None None] PATH_INFO: `/` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:53.752024 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 475/943] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 26 10:52:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 26 10:52:53.758593 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c530dd27-ea0a-4bd4-afa9-58c678ff46a8 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:53.759136 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c530dd27-ea0a-4bd4-afa9-58c678ff46a8 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:53.759637 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c530dd27-ea0a-4bd4-afa9-58c678ff46a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:53.779673 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-c530dd27-ea0a-4bd4-afa9-58c678ff46a8 None None] MFA Rules not processed for user `ca155a48e2024c32af3fd2e19e5fac11`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 10:52:53.781385 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-c530dd27-ea0a-4bd4-afa9-58c678ff46a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 10:52:53.836673 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5a2a36-003a-48f2-8c5b-a2ddbec0102f None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:52:53.878232 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 476/944] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 26 10:52:53 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 26 10:52:53.887511 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3b9dd8-4626-47b0-86cb-0b7ce10fa0ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:52:53.888468 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3b9dd8-4626-47b0-86cb-0b7ce10fa0ba None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:53.889016 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3b9dd8-4626-47b0-86cb-0b7ce10fa0ba None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:53.889487 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3b9dd8-4626-47b0-86cb-0b7ce10fa0ba None admin] PATH_INFO: `/v3/projects` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:53.890483 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe3b9dd8-4626-47b0-86cb-0b7ce10fa0ba None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 10:52:53.891539 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe3b9dd8-4626-47b0-86cb-0b7ce10fa0ba None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:52:53.893396 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe3b9dd8-4626-47b0-86cb-0b7ce10fa0ba None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:52:53.906532 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 469/945] 10.222.0.40 () {56 vars in 1148 bytes} [Fri Aug 26 10:52:53 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 10:52:53.913215 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c530dd27-ea0a-4bd4-afa9-58c678ff46a8 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:52:53.958588 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:52:54.019847 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c530dd27-ea0a-4bd4-afa9-58c678ff46a8 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:52:54.060524 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9e3e93-7a53-4588-8f85-da23415d0d24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:52:54.061198 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9e3e93-7a53-4588-8f85-da23415d0d24 None admin] REQUEST_METHOD: `PUT` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:54.061485 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9e3e93-7a53-4588-8f85-da23415d0d24 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:54.061765 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9e3e93-7a53-4588-8f85-da23415d0d24 None admin] PATH_INFO: `/v3/projects/0e30ccf0c1204727b786876c141dbcd2/users/50be12ccbd274a8792615e9dd10b61ca/roles/3f7d20781437466aabcbf11bda248f3d` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:54.064036 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b9e3e93-7a53-4588-8f85-da23415d0d24 None admin] RBAC: Authorizing `identity:create_grant(project_id=0e30ccf0c1204727b786876c141dbcd2, user_id=50be12ccbd274a8792615e9dd10b61ca, role_id=3f7d20781437466aabcbf11bda248f3d)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:52:54.065480 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b9e3e93-7a53-4588-8f85-da23415d0d24 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:52:54.080014 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 477/946] 10.222.0.40 () {58 vars in 1439 bytes} [Fri Aug 26 10:52:53 2022] PUT /identity/v3/projects/0e30ccf0c1204727b786876c141dbcd2/users/50be12ccbd274a8792615e9dd10b61ca/roles/3f7d20781437466aabcbf11bda248f3d => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 10:52:54.192407 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3b9dd8-4626-47b0-86cb-0b7ce10fa0ba None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:52:54.266425 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:52:54.316032 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3b9dd8-4626-47b0-86cb-0b7ce10fa0ba None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:52:54.352936 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5fa0a22-fd25-4ab2-bcc2-e18682425aeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:52:54.353553 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5fa0a22-fd25-4ab2-bcc2-e18682425aeb None admin] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:54.353849 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5fa0a22-fd25-4ab2-bcc2-e18682425aeb None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:54.354142 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5fa0a22-fd25-4ab2-bcc2-e18682425aeb None admin] PATH_INFO: `/v3/projects` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:54.354840 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5fa0a22-fd25-4ab2-bcc2-e18682425aeb None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:52:54.356277 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5fa0a22-fd25-4ab2-bcc2-e18682425aeb None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:52:54.359177 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: WARNING keystone.server.flask.common [None req-c5fa0a22-fd25-4ab2-bcc2-e18682425aeb None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Aug 26 10:52:54.377629 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.sql.core [None req-c5fa0a22-fd25-4ab2-bcc2-e18682425aeb None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 26 10:52:54.377629 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Aug 26 10:52:54.377629 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [parameters: {'id': '95909f8dd47c48cc8aec18f9c48df388', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Aug 26 10:52:54.377629 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81423) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 26 10:52:54.395179 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application [None req-c5fa0a22-fd25-4ab2-bcc2-e18682425aeb None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 95909f8dd47c48cc8aec18f9c48df388.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 95909f8dd47c48cc8aec18f9c48df388. Aug 26 10:52:54.395179 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 10:52:54.395179 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 26 10:52:54.395179 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 26 10:52:54.395179 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 26 10:52:54.395179 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 26 10:52:54.395179 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 26 10:52:54.395179 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application result = self._query(query) Aug 26 10:52:54.395179 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 26 10:52:54.395179 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application conn.query(q) Aug 26 10:52:54.395179 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 26 10:52:54.395179 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 26 10:52:54.395179 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 26 10:52:54.396085 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application result.read() Aug 26 10:52:54.396085 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 26 10:52:54.396085 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 26 10:52:54.396085 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 26 10:52:54.396085 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 26 10:52:54.396085 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 26 10:52:54.396085 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 26 10:52:54.396085 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 26 10:52:54.396085 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 26 10:52:54.396085 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 26 10:52:54.396085 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Aug 26 10:52:54.396085 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 26 10:52:54.396085 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Aug 26 10:52:54.396085 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 10:52:54.396085 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 26 10:52:54.396085 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 26 10:52:54.396085 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Aug 26 10:52:54.396085 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return project_ref.to_dict() Aug 26 10:52:54.397077 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 26 10:52:54.397077 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application next(self.gen) Aug 26 10:52:54.397077 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 26 10:52:54.397077 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application yield resource Aug 26 10:52:54.397077 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 26 10:52:54.397077 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application next(self.gen) Aug 26 10:52:54.397077 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 26 10:52:54.397077 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application self.session.rollback() Aug 26 10:52:54.397077 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 26 10:52:54.397077 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application self.force_reraise() Aug 26 10:52:54.397077 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 26 10:52:54.397077 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application raise self.value Aug 26 10:52:54.397077 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 26 10:52:54.397077 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 26 10:52:54.397077 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 26 10:52:54.397077 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application session.commit() Aug 26 10:52:54.397077 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 26 10:52:54.398615 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 26 10:52:54.398615 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 26 10:52:54.398615 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application self._prepare_impl() Aug 26 10:52:54.398615 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 26 10:52:54.398615 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application self.session.flush() Aug 26 10:52:54.398615 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 26 10:52:54.398615 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application self._flush(objects) Aug 26 10:52:54.398615 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 26 10:52:54.398615 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 26 10:52:54.398615 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 26 10:52:54.398615 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application compat.raise_( Aug 26 10:52:54.398615 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 26 10:52:54.398615 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application raise exception Aug 26 10:52:54.398615 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 26 10:52:54.398615 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application flush_context.execute() Aug 26 10:52:54.398615 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 26 10:52:54.398615 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application rec.execute(self) Aug 26 10:52:54.399559 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 26 10:52:54.399559 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 26 10:52:54.399559 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 26 10:52:54.399559 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 26 10:52:54.399559 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Aug 26 10:52:54.399559 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application c = connection._execute_20( Aug 26 10:52:54.399559 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Aug 26 10:52:54.399559 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 26 10:52:54.399559 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Aug 26 10:52:54.399559 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 26 10:52:54.399559 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Aug 26 10:52:54.399559 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 26 10:52:54.399559 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Aug 26 10:52:54.399559 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 26 10:52:54.399559 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Aug 26 10:52:54.400342 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 26 10:52:54.400342 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 26 10:52:54.400342 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application raise exception Aug 26 10:52:54.400342 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 26 10:52:54.400342 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 26 10:52:54.400342 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 26 10:52:54.400342 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 26 10:52:54.400342 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 26 10:52:54.400342 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application result = self._query(query) Aug 26 10:52:54.400342 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 26 10:52:54.400342 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application conn.query(q) Aug 26 10:52:54.400342 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 26 10:52:54.400342 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 26 10:52:54.400342 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 26 10:52:54.400342 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application result.read() Aug 26 10:52:54.400342 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 26 10:52:54.400342 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 26 10:52:54.401204 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 26 10:52:54.401204 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 26 10:52:54.401204 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 26 10:52:54.401204 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 26 10:52:54.401204 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 26 10:52:54.401204 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 26 10:52:54.401204 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 26 10:52:54.401204 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Aug 26 10:52:54.401204 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application [parameters: {'id': '95909f8dd47c48cc8aec18f9c48df388', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Aug 26 10:52:54.401204 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 26 10:52:54.401204 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Aug 26 10:52:54.401204 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 26 10:52:54.401204 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Aug 26 10:52:54.401204 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 10:52:54.401204 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Aug 26 10:52:54.403426 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Aug 26 10:52:54.403426 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 26 10:52:54.403426 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 26 10:52:54.403426 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Aug 26 10:52:54.403426 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Aug 26 10:52:54.403426 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 26 10:52:54.403426 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Aug 26 10:52:54.403426 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 10:52:54.403426 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 10:52:54.403426 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 10:52:54.403426 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 10:52:54.403426 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 10:52:54.403426 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 10:52:54.403426 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 10:52:54.403426 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 10:52:54.403426 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 10:52:54.403426 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 10:52:54.405665 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 10:52:54.405665 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Aug 26 10:52:54.405665 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Aug 26 10:52:54.405665 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 10:52:54.405665 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 10:52:54.405665 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Aug 26 10:52:54.405665 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application raise exception.Conflict( Aug 26 10:52:54.405665 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 95909f8dd47c48cc8aec18f9c48df388. Aug 26 10:52:54.405665 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Aug 26 10:52:54.405665 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 470/947] 10.222.0.40 () {60 vars in 1140 bytes} [Fri Aug 26 10:52:54 2022] POST /identity/v3/projects => generated 262 bytes in 205 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Aug 26 10:52:54.407533 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9e3e93-7a53-4588-8f85-da23415d0d24 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:52:54.512016 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9e3e93-7a53-4588-8f85-da23415d0d24 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:52:54.513571 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:52:54.550278 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb1cad6-c5e3-4fda-8de7-f6669d467295 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:52:54.550916 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb1cad6-c5e3-4fda-8de7-f6669d467295 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:54.551198 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb1cad6-c5e3-4fda-8de7-f6669d467295 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:54.551474 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb1cad6-c5e3-4fda-8de7-f6669d467295 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:54.558515 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb1cad6-c5e3-4fda-8de7-f6669d467295 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:52:54.559965 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb1cad6-c5e3-4fda-8de7-f6669d467295 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:52:54.571785 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application [None req-5bb1cad6-c5e3-4fda-8de7-f6669d467295 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 10:52:54.571785 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 10:52:54.571785 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 10:52:54.571785 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 10:52:54.571785 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 10:52:54.571785 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 10:52:54.571785 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 10:52:54.571785 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 10:52:54.571785 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 10:52:54.571785 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 10:52:54.571785 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 10:52:54.571785 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 10:52:54.571785 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 26 10:52:54.571785 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 26 10:52:54.571785 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 26 10:52:54.572706 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 26 10:52:54.572706 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 10:52:54.572706 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 10:52:54.572706 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 10:52:54.572706 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 10:52:54.572706 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 10:52:54.572706 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 10:52:54.572706 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 10:52:54.572706 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application with Lock( Aug 26 10:52:54.572706 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 10:52:54.572706 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._enter() Aug 26 10:52:54.572706 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 10:52:54.572706 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 10:52:54.572706 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 10:52:54.572706 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.creator() Aug 26 10:52:54.572706 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 10:52:54.572706 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application created_value = creator( Aug 26 10:52:54.573628 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 26 10:52:54.573628 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 26 10:52:54.573628 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 26 10:52:54.573628 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 26 10:52:54.573628 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 26 10:52:54.573628 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 26 10:52:54.573628 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 10:52:54.573628 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Aug 26 10:52:54.573628 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 478/948] 10.222.0.40 () {56 vars in 1111 bytes} [Fri Aug 26 10:52:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 10:52:54.580024 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5fa0a22-fd25-4ab2-bcc2-e18682425aeb None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:52:54.687638 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5fa0a22-fd25-4ab2-bcc2-e18682425aeb None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:52:54.706787 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:52:54.717623 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9ef3ab-1dc1-4a73-86ea-bf70ac0ab5e9 None None] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:54.718154 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9ef3ab-1dc1-4a73-86ea-bf70ac0ab5e9 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:54.718414 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9ef3ab-1dc1-4a73-86ea-bf70ac0ab5e9 None None] PATH_INFO: `/` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:54.720480 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 479/949] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 26 10:52:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 26 10:52:54.725694 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa9e10b-73ea-470a-888b-acacd117322d None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:54.726278 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa9e10b-73ea-470a-888b-acacd117322d None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:54.726751 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa9e10b-73ea-470a-888b-acacd117322d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:54.729064 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8832870c-436d-4314-bb2b-30a272a03ced None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:52:54.729680 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8832870c-436d-4314-bb2b-30a272a03ced None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:54.729963 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8832870c-436d-4314-bb2b-30a272a03ced None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:54.730279 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8832870c-436d-4314-bb2b-30a272a03ced None admin] PATH_INFO: `/v3/projects` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:54.730958 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8832870c-436d-4314-bb2b-30a272a03ced None admin] RBAC: Adding query filter params (name=service) {{(pid=81423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 10:52:54.731286 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8832870c-436d-4314-bb2b-30a272a03ced None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:52:54.733152 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8832870c-436d-4314-bb2b-30a272a03ced None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:52:54.748103 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-ffa9e10b-73ea-470a-888b-acacd117322d None None] MFA Rules not processed for user `ca155a48e2024c32af3fd2e19e5fac11`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 10:52:54.749593 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-ffa9e10b-73ea-470a-888b-acacd117322d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 10:52:54.758484 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 471/950] 10.222.0.40 () {56 vars in 1112 bytes} [Fri Aug 26 10:52:54 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 10:52:54.836166 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 480/951] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 26 10:52:54 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 26 10:52:55.104674 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8832870c-436d-4314-bb2b-30a272a03ced None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:52:55.105133 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2017b4-ab62-4c0c-8088-d4bad76cf0b9 None None] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:55.105418 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2017b4-ab62-4c0c-8088-d4bad76cf0b9 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:55.105690 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2017b4-ab62-4c0c-8088-d4bad76cf0b9 None None] PATH_INFO: `/` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:55.107360 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 481/952] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 26 10:52:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 26 10:52:55.113469 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e63fc92-28d7-4ce1-aafe-e1039e26ab72 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:55.113751 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e63fc92-28d7-4ce1-aafe-e1039e26ab72 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:55.114050 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e63fc92-28d7-4ce1-aafe-e1039e26ab72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:55.138056 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-4e63fc92-28d7-4ce1-aafe-e1039e26ab72 None None] MFA Rules not processed for user `ca155a48e2024c32af3fd2e19e5fac11`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 10:52:55.139974 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-4e63fc92-28d7-4ce1-aafe-e1039e26ab72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 10:52:55.226855 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8832870c-436d-4314-bb2b-30a272a03ced None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:52:55.243887 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 482/953] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 26 10:52:55 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 26 10:52:55.268455 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48540b27-ad88-400b-8bd1-cb2a29926c51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:52:55.269087 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48540b27-ad88-400b-8bd1-cb2a29926c51 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:55.269360 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48540b27-ad88-400b-8bd1-cb2a29926c51 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:55.269623 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48540b27-ad88-400b-8bd1-cb2a29926c51 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:55.270231 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48540b27-ad88-400b-8bd1-cb2a29926c51 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:52:55.271641 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48540b27-ad88-400b-8bd1-cb2a29926c51 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:52:55.280540 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application [None req-48540b27-ad88-400b-8bd1-cb2a29926c51 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 26 10:52:55.280540 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 10:52:55.280540 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 26 10:52:55.280540 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 26 10:52:55.280540 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 26 10:52:55.280540 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 26 10:52:55.280540 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 26 10:52:55.280540 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 26 10:52:55.280540 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 26 10:52:55.280540 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Aug 26 10:52:55.280540 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 26 10:52:55.280540 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Aug 26 10:52:55.280540 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 10:52:55.280540 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 10:52:55.280540 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 10:52:55.280540 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 10:52:55.280540 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 10:52:55.281506 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 10:52:55.281506 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 10:52:55.281506 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 10:52:55.281506 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 10:52:55.281506 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 10:52:55.281506 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 10:52:55.281506 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 26 10:52:55.281506 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 26 10:52:55.281506 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 26 10:52:55.281506 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 26 10:52:55.281506 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 10:52:55.281506 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 10:52:55.281506 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 10:52:55.281506 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 10:52:55.281506 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 10:52:55.281506 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 10:52:55.283279 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 10:52:55.283279 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application with Lock( Aug 26 10:52:55.283279 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 10:52:55.283279 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._enter() Aug 26 10:52:55.283279 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 10:52:55.283279 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 10:52:55.283279 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 10:52:55.283279 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.creator() Aug 26 10:52:55.283279 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 10:52:55.283279 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application created_value = creator( Aug 26 10:52:55.283279 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 26 10:52:55.283279 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 26 10:52:55.283279 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 26 10:52:55.283279 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Aug 26 10:52:55.283279 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 472/954] 10.222.0.40 () {56 vars in 1126 bytes} [Fri Aug 26 10:52:55 2022] GET /identity/v3/services/object-store => generated 93 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 10:52:55.292409 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e63fc92-28d7-4ce1-aafe-e1039e26ab72 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:52:55.321052 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:52:55.403624 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e63fc92-28d7-4ce1-aafe-e1039e26ab72 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:52:55.439401 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbec72b3-99f2-464f-9bd3-27e70c826030 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:52:55.440013 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbec72b3-99f2-464f-9bd3-27e70c826030 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:55.440361 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbec72b3-99f2-464f-9bd3-27e70c826030 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:55.440654 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbec72b3-99f2-464f-9bd3-27e70c826030 None admin] PATH_INFO: `/v3/services` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:55.441271 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbec72b3-99f2-464f-9bd3-27e70c826030 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 10:52:55.441568 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbec72b3-99f2-464f-9bd3-27e70c826030 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:52:55.442942 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbec72b3-99f2-464f-9bd3-27e70c826030 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:52:55.448025 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 483/955] 10.222.0.40 () {56 vars in 1122 bytes} [Fri Aug 26 10:52:55 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 10:52:55.454732 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48540b27-ad88-400b-8bd1-cb2a29926c51 None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:52:55.493281 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:52:55.519351 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbec72b3-99f2-464f-9bd3-27e70c826030 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:52:55.564579 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48540b27-ad88-400b-8bd1-cb2a29926c51 None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:52:55.613624 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22fd6e16-9a01-42a3-bd09-40f7272def1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:52:55.614264 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fd6e16-9a01-42a3-bd09-40f7272def1f None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:55.614537 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fd6e16-9a01-42a3-bd09-40f7272def1f None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:55.614804 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fd6e16-9a01-42a3-bd09-40f7272def1f None admin] PATH_INFO: `/v3/services` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:55.615432 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22fd6e16-9a01-42a3-bd09-40f7272def1f None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 10:52:55.615725 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22fd6e16-9a01-42a3-bd09-40f7272def1f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:52:55.617617 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22fd6e16-9a01-42a3-bd09-40f7272def1f None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:52:55.626333 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 473/956] 10.222.0.40 () {56 vars in 1122 bytes} [Fri Aug 26 10:52:55 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 10:52:55.632223 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:52:55.633356 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22fd6e16-9a01-42a3-bd09-40f7272def1f None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:52:55.650856 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbec72b3-99f2-464f-9bd3-27e70c826030 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:52:55.687704 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11643b5-1355-4769-ae02-139d95a161c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:52:55.689470 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11643b5-1355-4769-ae02-139d95a161c7 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:55.689666 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11643b5-1355-4769-ae02-139d95a161c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:55.690001 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11643b5-1355-4769-ae02-139d95a161c7 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:55.690625 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c11643b5-1355-4769-ae02-139d95a161c7 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:52:55.692221 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c11643b5-1355-4769-ae02-139d95a161c7 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:52:55.705325 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application [None req-c11643b5-1355-4769-ae02-139d95a161c7 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 26 10:52:55.705325 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 10:52:55.705325 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 26 10:52:55.705325 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 26 10:52:55.705325 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 26 10:52:55.705325 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 26 10:52:55.705325 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 26 10:52:55.705325 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 26 10:52:55.705325 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 26 10:52:55.705325 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Aug 26 10:52:55.705325 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 26 10:52:55.705325 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Aug 26 10:52:55.705325 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 10:52:55.705325 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 10:52:55.705325 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 10:52:55.705325 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 10:52:55.705325 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 10:52:55.706707 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 10:52:55.706707 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 10:52:55.706707 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 10:52:55.706707 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 10:52:55.706707 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 10:52:55.706707 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 10:52:55.706707 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 26 10:52:55.706707 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 26 10:52:55.706707 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 26 10:52:55.706707 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 26 10:52:55.706707 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 10:52:55.706707 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 10:52:55.706707 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 10:52:55.706707 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 10:52:55.706707 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 10:52:55.706707 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 10:52:55.712525 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 10:52:55.712525 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application with Lock( Aug 26 10:52:55.712525 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 10:52:55.712525 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._enter() Aug 26 10:52:55.712525 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 10:52:55.712525 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 10:52:55.712525 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 10:52:55.712525 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.creator() Aug 26 10:52:55.712525 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 10:52:55.712525 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application created_value = creator( Aug 26 10:52:55.712525 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 26 10:52:55.712525 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 26 10:52:55.712525 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 26 10:52:55.712525 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Aug 26 10:52:55.712525 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 484/957] 10.222.0.40 () {56 vars in 1111 bytes} [Fri Aug 26 10:52:55 2022] GET /identity/v3/services/compute => generated 88 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 26 10:52:55.715420 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11643b5-1355-4769-ae02-139d95a161c7 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:52:55.767829 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:52:55.771152 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22fd6e16-9a01-42a3-bd09-40f7272def1f None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:52:55.813883 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c8b31a-8f69-4b71-98bb-5d37b4000104 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:52:55.814761 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c8b31a-8f69-4b71-98bb-5d37b4000104 None admin] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:55.815258 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c8b31a-8f69-4b71-98bb-5d37b4000104 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:55.815528 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c8b31a-8f69-4b71-98bb-5d37b4000104 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:55.816186 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c8b31a-8f69-4b71-98bb-5d37b4000104 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:52:55.817543 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c8b31a-8f69-4b71-98bb-5d37b4000104 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:52:55.846005 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 474/958] 10.222.0.40 () {60 vars in 1144 bytes} [Fri Aug 26 10:52:55 2022] POST /identity/v3/endpoints => generated 349 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 10:52:55.850582 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11643b5-1355-4769-ae02-139d95a161c7 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:52:55.906247 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c1358e-74ea-482e-8762-cdf28eaffe92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:52:55.907241 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c1358e-74ea-482e-8762-cdf28eaffe92 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:55.907699 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c1358e-74ea-482e-8762-cdf28eaffe92 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:55.908173 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c1358e-74ea-482e-8762-cdf28eaffe92 None admin] PATH_INFO: `/v3/services` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:55.908993 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c1358e-74ea-482e-8762-cdf28eaffe92 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 10:52:55.909482 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c1358e-74ea-482e-8762-cdf28eaffe92 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:52:55.911117 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c1358e-74ea-482e-8762-cdf28eaffe92 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:52:55.920913 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 485/959] 10.222.0.40 () {56 vars in 1112 bytes} [Fri Aug 26 10:52:55 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 10:52:55.924670 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c8b31a-8f69-4b71-98bb-5d37b4000104 None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:52:56.040274 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c8b31a-8f69-4b71-98bb-5d37b4000104 None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:52:56.078032 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291da41e-0b5c-44df-911b-54140cf3e40a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:52:56.079476 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291da41e-0b5c-44df-911b-54140cf3e40a None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:56.079936 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291da41e-0b5c-44df-911b-54140cf3e40a None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:56.080406 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291da41e-0b5c-44df-911b-54140cf3e40a None admin] PATH_INFO: `/v3/services` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:56.081212 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-291da41e-0b5c-44df-911b-54140cf3e40a None admin] RBAC: Adding query filter params (type=compute) {{(pid=81423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 10:52:56.081688 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-291da41e-0b5c-44df-911b-54140cf3e40a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:52:56.083257 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-291da41e-0b5c-44df-911b-54140cf3e40a None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:52:56.094762 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 475/960] 10.222.0.40 () {56 vars in 1112 bytes} [Fri Aug 26 10:52:55 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 10:52:56.182748 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146b4678-0069-4ea2-a9fe-cd3b43030f79 None None] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:56.183400 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146b4678-0069-4ea2-a9fe-cd3b43030f79 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:56.183896 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146b4678-0069-4ea2-a9fe-cd3b43030f79 None None] PATH_INFO: `/` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:56.185835 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 486/961] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 26 10:52:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 10:52:56.191198 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d003c6-d9ca-4f9d-965e-382e2591a5b4 None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:56.191757 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d003c6-d9ca-4f9d-965e-382e2591a5b4 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:56.192260 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d003c6-d9ca-4f9d-965e-382e2591a5b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:56.213201 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-f0d003c6-d9ca-4f9d-965e-382e2591a5b4 None None] MFA Rules not processed for user `ca155a48e2024c32af3fd2e19e5fac11`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 10:52:56.215281 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-f0d003c6-d9ca-4f9d-965e-382e2591a5b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 10:52:56.234859 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:52:56.295662 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 476/962] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 26 10:52:56 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 26 10:52:56.565108 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146b4678-0069-4ea2-a9fe-cd3b43030f79 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:52:56.577362 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:52:56.671427 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146b4678-0069-4ea2-a9fe-cd3b43030f79 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:52:56.707713 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb841d6e-7af9-46db-a4ff-bf6d1f4b47ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:52:56.708345 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb841d6e-7af9-46db-a4ff-bf6d1f4b47ed None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:56.708623 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb841d6e-7af9-46db-a4ff-bf6d1f4b47ed None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:56.708894 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb841d6e-7af9-46db-a4ff-bf6d1f4b47ed None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:56.719500 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb841d6e-7af9-46db-a4ff-bf6d1f4b47ed None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 10:52:56.724475 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application [None req-cb841d6e-7af9-46db-a4ff-bf6d1f4b47ed None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 26 10:52:56.724475 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 10:52:56.724475 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 10:52:56.724475 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 10:52:56.724475 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 10:52:56.724475 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 10:52:56.724475 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 10:52:56.724475 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 10:52:56.724475 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 10:52:56.724475 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 10:52:56.724475 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 10:52:56.724475 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 10:52:56.724475 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 10:52:56.724475 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 10:52:56.724475 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 10:52:56.724475 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 10:52:56.725401 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 10:52:56.725401 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 10:52:56.725401 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 10:52:56.725401 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 10:52:56.725401 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 10:52:56.725401 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 10:52:56.725401 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 10:52:56.725401 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 10:52:56.725401 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 10:52:56.725401 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 10:52:56.725401 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 10:52:56.725401 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application with Lock( Aug 26 10:52:56.725401 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 10:52:56.725401 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._enter() Aug 26 10:52:56.725401 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 10:52:56.725401 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 10:52:56.726285 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 10:52:56.726285 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.creator() Aug 26 10:52:56.726285 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 10:52:56.726285 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application created_value = creator( Aug 26 10:52:56.726285 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 10:52:56.726285 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 10:52:56.726285 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 10:52:56.726285 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 10:52:56.726285 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 10:52:56.726285 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 10:52:56.726285 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 26 10:52:56.726285 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Aug 26 10:52:56.726285 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 487/963] 10.222.0.40 () {56 vars in 1120 bytes} [Fri Aug 26 10:52:56 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 26 10:52:56.731854 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d003c6-d9ca-4f9d-965e-382e2591a5b4 None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:52:56.789344 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921a4dae-c2b4-4545-ae76-c3c17372212c None None] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:56.789669 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921a4dae-c2b4-4545-ae76-c3c17372212c None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:56.789903 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921a4dae-c2b4-4545-ae76-c3c17372212c None None] PATH_INFO: `/` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:56.792466 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 488/964] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 26 10:52:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 26 10:52:56.796589 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd39a992-eae6-491a-bf49-1c7e3b21c887 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:56.796873 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd39a992-eae6-491a-bf49-1c7e3b21c887 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:56.797209 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd39a992-eae6-491a-bf49-1c7e3b21c887 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:56.818171 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-bd39a992-eae6-491a-bf49-1c7e3b21c887 None None] MFA Rules not processed for user `ca155a48e2024c32af3fd2e19e5fac11`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 10:52:56.819551 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-bd39a992-eae6-491a-bf49-1c7e3b21c887 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 10:52:56.840214 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d003c6-d9ca-4f9d-965e-382e2591a5b4 None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:52:56.888001 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c31211d-f491-4ac0-9038-a28f82fc14ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:52:56.888626 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c31211d-f491-4ac0-9038-a28f82fc14ce None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:56.888905 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c31211d-f491-4ac0-9038-a28f82fc14ce None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:56.889175 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c31211d-f491-4ac0-9038-a28f82fc14ce None admin] PATH_INFO: `/v3/roles` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:56.889846 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c31211d-f491-4ac0-9038-a28f82fc14ce None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 10:52:56.890193 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c31211d-f491-4ac0-9038-a28f82fc14ce None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:52:56.892332 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c31211d-f491-4ac0-9038-a28f82fc14ce None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:52:56.902265 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 477/965] 10.222.0.40 () {56 vars in 1115 bytes} [Fri Aug 26 10:52:56 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 10:52:56.908191 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c31211d-f491-4ac0-9038-a28f82fc14ce None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:52:56.913824 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:52:56.926581 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 489/966] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 26 10:52:56 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 26 10:52:56.927012 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:52:57.016874 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c31211d-f491-4ac0-9038-a28f82fc14ce None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:52:57.054587 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff46671-ba7f-49e5-9b6f-8fc9397ad1ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:52:57.055322 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff46671-ba7f-49e5-9b6f-8fc9397ad1ec None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:57.055600 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff46671-ba7f-49e5-9b6f-8fc9397ad1ec None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:57.055871 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff46671-ba7f-49e5-9b6f-8fc9397ad1ec None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:57.068319 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff46671-ba7f-49e5-9b6f-8fc9397ad1ec None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:52:57.069760 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff46671-ba7f-49e5-9b6f-8fc9397ad1ec None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:52:57.084803 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application [None req-1ff46671-ba7f-49e5-9b6f-8fc9397ad1ec None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 10:52:57.084803 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 10:52:57.084803 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 10:52:57.084803 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 10:52:57.084803 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 10:52:57.084803 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 10:52:57.084803 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 10:52:57.084803 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 10:52:57.084803 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 10:52:57.084803 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 10:52:57.084803 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 10:52:57.084803 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 10:52:57.084803 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 10:52:57.084803 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 10:52:57.084803 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 10:52:57.084803 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 10:52:57.085738 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 10:52:57.085738 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 10:52:57.085738 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 10:52:57.085738 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 10:52:57.085738 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 10:52:57.085738 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 10:52:57.085738 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 10:52:57.085738 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application with Lock( Aug 26 10:52:57.085738 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 10:52:57.085738 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._enter() Aug 26 10:52:57.085738 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 10:52:57.085738 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 10:52:57.085738 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 10:52:57.085738 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.creator() Aug 26 10:52:57.085738 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 10:52:57.085738 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application created_value = creator( Aug 26 10:52:57.085738 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 10:52:57.086706 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 10:52:57.086706 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 10:52:57.086706 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Aug 26 10:52:57.089487 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 478/967] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Aug 26 10:52:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 26 10:52:57.092228 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd39a992-eae6-491a-bf49-1c7e3b21c887 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:52:57.197161 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd39a992-eae6-491a-bf49-1c7e3b21c887 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:52:57.211401 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff46671-ba7f-49e5-9b6f-8fc9397ad1ec None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:52:57.238218 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95549fdb-5102-428a-8d53-998e48e59dec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:52:57.238837 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95549fdb-5102-428a-8d53-998e48e59dec None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:57.239125 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95549fdb-5102-428a-8d53-998e48e59dec None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:57.239397 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95549fdb-5102-428a-8d53-998e48e59dec None admin] PATH_INFO: `/v3/domains` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:57.240410 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95549fdb-5102-428a-8d53-998e48e59dec None admin] RBAC: Adding query filter params (name=Default) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 10:52:57.240767 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95549fdb-5102-428a-8d53-998e48e59dec None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:52:57.242394 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95549fdb-5102-428a-8d53-998e48e59dec None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:52:57.262082 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 490/968] 10.222.0.40 () {56 vars in 1109 bytes} [Fri Aug 26 10:52:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 10:52:57.266529 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95549fdb-5102-428a-8d53-998e48e59dec None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:52:57.289328 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:52:57.331878 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff46671-ba7f-49e5-9b6f-8fc9397ad1ec None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:52:57.358485 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:52:57.370305 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e401be1-196b-4741-96a7-7ff2bcc4f6cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:52:57.371216 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e401be1-196b-4741-96a7-7ff2bcc4f6cb None admin] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:57.371718 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e401be1-196b-4741-96a7-7ff2bcc4f6cb None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:57.372184 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e401be1-196b-4741-96a7-7ff2bcc4f6cb None admin] PATH_INFO: `/v3/users` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:57.373045 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e401be1-196b-4741-96a7-7ff2bcc4f6cb None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:52:57.374706 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e401be1-196b-4741-96a7-7ff2bcc4f6cb None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:52:57.392413 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.sql.core [None req-3e401be1-196b-4741-96a7-7ff2bcc4f6cb None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 26 10:52:57.392413 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Aug 26 10:52:57.392413 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [parameters: {'user_id': 'ea505ec73e57490c999f262adc1d5b88', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Aug 26 10:52:57.392413 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81423) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 26 10:52:57.392811 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95549fdb-5102-428a-8d53-998e48e59dec None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:52:57.402630 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application [None req-3e401be1-196b-4741-96a7-7ff2bcc4f6cb None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Aug 26 10:52:57.402630 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 10:52:57.402630 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 26 10:52:57.402630 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 26 10:52:57.402630 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 26 10:52:57.402630 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 26 10:52:57.402630 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 26 10:52:57.402630 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application result = self._query(query) Aug 26 10:52:57.402630 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 26 10:52:57.402630 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application conn.query(q) Aug 26 10:52:57.402630 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 26 10:52:57.402630 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 26 10:52:57.402630 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 26 10:52:57.402630 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application result.read() Aug 26 10:52:57.402630 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 26 10:52:57.403545 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 26 10:52:57.403545 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 26 10:52:57.403545 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 26 10:52:57.403545 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 26 10:52:57.403545 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 26 10:52:57.403545 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 26 10:52:57.403545 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 26 10:52:57.403545 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 26 10:52:57.403545 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Aug 26 10:52:57.403545 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 26 10:52:57.403545 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Aug 26 10:52:57.403545 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 10:52:57.403545 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 26 10:52:57.403545 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 26 10:52:57.403545 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Aug 26 10:52:57.403545 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Aug 26 10:52:57.403545 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 26 10:52:57.403545 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application next(self.gen) Aug 26 10:52:57.407244 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 26 10:52:57.407244 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application yield resource Aug 26 10:52:57.407244 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 26 10:52:57.407244 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application next(self.gen) Aug 26 10:52:57.407244 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 26 10:52:57.407244 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application self.session.rollback() Aug 26 10:52:57.407244 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 26 10:52:57.407244 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application self.force_reraise() Aug 26 10:52:57.407244 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 26 10:52:57.407244 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application raise self.value Aug 26 10:52:57.407244 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 26 10:52:57.407244 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 26 10:52:57.407244 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 26 10:52:57.407244 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application session.commit() Aug 26 10:52:57.407244 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 26 10:52:57.407244 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 26 10:52:57.407244 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 26 10:52:57.408209 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application self._prepare_impl() Aug 26 10:52:57.408209 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 26 10:52:57.408209 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application self.session.flush() Aug 26 10:52:57.408209 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 26 10:52:57.408209 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application self._flush(objects) Aug 26 10:52:57.408209 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 26 10:52:57.408209 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 26 10:52:57.408209 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 26 10:52:57.408209 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application compat.raise_( Aug 26 10:52:57.408209 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 26 10:52:57.408209 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application raise exception Aug 26 10:52:57.408209 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 26 10:52:57.408209 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application flush_context.execute() Aug 26 10:52:57.408209 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 26 10:52:57.408209 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application rec.execute(self) Aug 26 10:52:57.408209 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 26 10:52:57.408209 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 26 10:52:57.409176 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 26 10:52:57.409176 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 26 10:52:57.409176 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Aug 26 10:52:57.409176 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application result = connection._execute_20( Aug 26 10:52:57.409176 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Aug 26 10:52:57.409176 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 26 10:52:57.409176 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Aug 26 10:52:57.409176 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 26 10:52:57.409176 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Aug 26 10:52:57.409176 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 26 10:52:57.409176 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Aug 26 10:52:57.409176 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 26 10:52:57.409176 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Aug 26 10:52:57.409176 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 26 10:52:57.409176 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 26 10:52:57.409176 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application raise exception Aug 26 10:52:57.410054 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 26 10:52:57.410054 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 26 10:52:57.410054 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 26 10:52:57.410054 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 26 10:52:57.410054 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 26 10:52:57.410054 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application result = self._query(query) Aug 26 10:52:57.410054 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 26 10:52:57.410054 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application conn.query(q) Aug 26 10:52:57.410054 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 26 10:52:57.410054 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 26 10:52:57.410054 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 26 10:52:57.410054 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application result.read() Aug 26 10:52:57.410054 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 26 10:52:57.410054 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 26 10:52:57.410054 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 26 10:52:57.410054 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 26 10:52:57.410957 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 26 10:52:57.410957 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 26 10:52:57.410957 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 26 10:52:57.410957 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 26 10:52:57.410957 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 26 10:52:57.410957 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Aug 26 10:52:57.410957 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application [parameters: {'user_id': 'ea505ec73e57490c999f262adc1d5b88', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Aug 26 10:52:57.410957 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 26 10:52:57.410957 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Aug 26 10:52:57.410957 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 26 10:52:57.410957 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Aug 26 10:52:57.410957 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 10:52:57.410957 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 10:52:57.410957 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 10:52:57.410957 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 10:52:57.411758 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 10:52:57.411758 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 10:52:57.411758 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 10:52:57.411758 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 10:52:57.411758 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 10:52:57.411758 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 10:52:57.411758 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 10:52:57.411758 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Aug 26 10:52:57.411758 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Aug 26 10:52:57.411758 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 10:52:57.411758 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 10:52:57.411758 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 26 10:52:57.411758 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 10:52:57.411758 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 26 10:52:57.411758 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 10:52:57.411758 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Aug 26 10:52:57.411758 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Aug 26 10:52:57.412875 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Aug 26 10:52:57.412875 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Aug 26 10:52:57.412875 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 26 10:52:57.412875 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 26 10:52:57.412875 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Aug 26 10:52:57.412875 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Aug 26 10:52:57.412875 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 479/969] 10.222.0.40 () {60 vars in 1131 bytes} [Fri Aug 26 10:52:57 2022] POST /identity/v3/users => generated 160 bytes in 194 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Aug 26 10:52:57.412875 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e401be1-196b-4741-96a7-7ff2bcc4f6cb None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:52:57.440481 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28ad44a-d3a1-452a-a722-c987823076a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:52:57.441413 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28ad44a-d3a1-452a-a722-c987823076a7 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:57.441895 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28ad44a-d3a1-452a-a722-c987823076a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:57.442498 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28ad44a-d3a1-452a-a722-c987823076a7 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:57.452122 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b28ad44a-d3a1-452a-a722-c987823076a7 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:52:57.453951 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b28ad44a-d3a1-452a-a722-c987823076a7 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:52:57.465993 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application [None req-b28ad44a-d3a1-452a-a722-c987823076a7 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 26 10:52:57.465993 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 10:52:57.465993 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 10:52:57.465993 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 10:52:57.465993 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 10:52:57.465993 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 10:52:57.465993 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 10:52:57.465993 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 10:52:57.465993 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 10:52:57.465993 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 10:52:57.465993 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 10:52:57.465993 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 10:52:57.465993 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 26 10:52:57.465993 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 26 10:52:57.465993 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 26 10:52:57.465993 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 26 10:52:57.467027 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 10:52:57.467027 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 10:52:57.467027 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 26 10:52:57.467027 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 10:52:57.467027 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 26 10:52:57.467027 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 10:52:57.467027 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 10:52:57.467027 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 10:52:57.467027 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 10:52:57.467027 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 10:52:57.467027 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 10:52:57.467027 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application with Lock( Aug 26 10:52:57.467027 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 10:52:57.467027 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._enter() Aug 26 10:52:57.467027 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 10:52:57.467027 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 10:52:57.467027 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 10:52:57.467027 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.creator() Aug 26 10:52:57.467977 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 10:52:57.467977 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application created_value = creator( Aug 26 10:52:57.467977 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 26 10:52:57.467977 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 26 10:52:57.467977 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 26 10:52:57.467977 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 26 10:52:57.467977 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 26 10:52:57.467977 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 26 10:52:57.467977 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 26 10:52:57.467977 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Aug 26 10:52:57.475265 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 491/970] 10.222.0.40 () {56 vars in 1117 bytes} [Fri Aug 26 10:52:57 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 26 10:52:57.476364 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28ad44a-d3a1-452a-a722-c987823076a7 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:52:57.541824 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e401be1-196b-4741-96a7-7ff2bcc4f6cb None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:52:57.572063 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:52:57.584018 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553d8624-b642-465e-ac8d-07144d1e5be8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:52:57.585056 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553d8624-b642-465e-ac8d-07144d1e5be8 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:57.585531 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553d8624-b642-465e-ac8d-07144d1e5be8 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:57.586098 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553d8624-b642-465e-ac8d-07144d1e5be8 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:57.595603 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-553d8624-b642-465e-ac8d-07144d1e5be8 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:52:57.597829 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-553d8624-b642-465e-ac8d-07144d1e5be8 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:52:57.604282 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28ad44a-d3a1-452a-a722-c987823076a7 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:52:57.611569 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application [None req-553d8624-b642-465e-ac8d-07144d1e5be8 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 26 10:52:57.611569 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 10:52:57.611569 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 10:52:57.611569 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 10:52:57.611569 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 10:52:57.611569 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 10:52:57.611569 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 10:52:57.611569 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 10:52:57.611569 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 10:52:57.611569 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 10:52:57.611569 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 10:52:57.611569 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 10:52:57.611569 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 26 10:52:57.611569 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 26 10:52:57.611569 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 26 10:52:57.611569 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 26 10:52:57.612552 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 10:52:57.612552 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 10:52:57.612552 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 26 10:52:57.612552 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 10:52:57.612552 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 26 10:52:57.612552 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 10:52:57.612552 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 10:52:57.612552 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 10:52:57.612552 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 10:52:57.612552 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 10:52:57.612552 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 10:52:57.612552 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application with Lock( Aug 26 10:52:57.612552 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 10:52:57.612552 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._enter() Aug 26 10:52:57.612552 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 10:52:57.612552 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 10:52:57.612552 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 10:52:57.612552 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.creator() Aug 26 10:52:57.613506 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 10:52:57.613506 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application created_value = creator( Aug 26 10:52:57.613506 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 26 10:52:57.613506 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 26 10:52:57.613506 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 26 10:52:57.613506 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 26 10:52:57.613506 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 26 10:52:57.613506 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 26 10:52:57.613506 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 26 10:52:57.613506 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Aug 26 10:52:57.616158 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 480/971] 10.222.0.40 () {56 vars in 1099 bytes} [Fri Aug 26 10:52:57 2022] GET /identity/v3/users/cinder => generated 84 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 10:52:57.623259 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553d8624-b642-465e-ac8d-07144d1e5be8 None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:52:57.630459 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:52:57.647921 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c7a5fd-ed1a-4086-830d-ee6600c167d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:52:57.648916 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c7a5fd-ed1a-4086-830d-ee6600c167d7 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:57.649423 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c7a5fd-ed1a-4086-830d-ee6600c167d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:57.649897 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c7a5fd-ed1a-4086-830d-ee6600c167d7 None admin] PATH_INFO: `/v3/users` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:57.650861 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c7a5fd-ed1a-4086-830d-ee6600c167d7 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 10:52:57.651371 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c7a5fd-ed1a-4086-830d-ee6600c167d7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:52:57.653111 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c7a5fd-ed1a-4086-830d-ee6600c167d7 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:52:57.678971 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 492/972] 10.222.0.40 () {56 vars in 1149 bytes} [Fri Aug 26 10:52:57 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 10:52:57.680215 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c7a5fd-ed1a-4086-830d-ee6600c167d7 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:52:57.738518 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553d8624-b642-465e-ac8d-07144d1e5be8 None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:52:57.776720 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02aa9e3-e3f4-4a3b-9232-f40935525a49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:52:57.777338 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02aa9e3-e3f4-4a3b-9232-f40935525a49 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:57.777618 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02aa9e3-e3f4-4a3b-9232-f40935525a49 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:57.777885 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02aa9e3-e3f4-4a3b-9232-f40935525a49 None admin] PATH_INFO: `/v3/users` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:57.778616 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a02aa9e3-e3f4-4a3b-9232-f40935525a49 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=81423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 10:52:57.778912 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a02aa9e3-e3f4-4a3b-9232-f40935525a49 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:52:57.780324 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a02aa9e3-e3f4-4a3b-9232-f40935525a49 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:52:57.790743 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c7a5fd-ed1a-4086-830d-ee6600c167d7 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:52:57.826082 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 481/973] 10.222.0.40 () {56 vars in 1101 bytes} [Fri Aug 26 10:52:57 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 10:52:57.830668 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12880d38-d5b1-4fd8-85eb-677dd644b18a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:52:57.831624 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12880d38-d5b1-4fd8-85eb-677dd644b18a None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:57.832157 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12880d38-d5b1-4fd8-85eb-677dd644b18a None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:57.832698 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12880d38-d5b1-4fd8-85eb-677dd644b18a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:57.893677 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:52:57.923335 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4399840-9b63-4c12-92d2-a27955af23bb None None] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:57.923893 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12880d38-d5b1-4fd8-85eb-677dd644b18a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:52:57.924243 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4399840-9b63-4c12-92d2-a27955af23bb None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:57.924243 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4399840-9b63-4c12-92d2-a27955af23bb None None] PATH_INFO: `/` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:57.924846 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12880d38-d5b1-4fd8-85eb-677dd644b18a None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:52:57.926052 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 482/974] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 26 10:52:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 10:52:57.938505 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2b63e0-b44f-47bc-82a3-8c6f2facb678 None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:57.938981 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2b63e0-b44f-47bc-82a3-8c6f2facb678 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:57.939356 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2b63e0-b44f-47bc-82a3-8c6f2facb678 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:57.967006 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-7e2b63e0-b44f-47bc-82a3-8c6f2facb678 None None] MFA Rules not processed for user `ca155a48e2024c32af3fd2e19e5fac11`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 10:52:57.968571 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-7e2b63e0-b44f-47bc-82a3-8c6f2facb678 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 10:52:58.023180 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:52:58.046294 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 493/975] 10.222.0.40 () {58 vars in 1282 bytes} [Fri Aug 26 10:52:57 2022] GET /identity/v3/auth/tokens => generated 1474 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 26 10:52:58.053618 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12880d38-d5b1-4fd8-85eb-677dd644b18a None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:52:58.070338 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 483/976] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 26 10:52:57 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 26 10:52:58.108869 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a3c87d-b0e5-464b-a910-7c5ec81517dd None None] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:58.109163 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a3c87d-b0e5-464b-a910-7c5ec81517dd None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:58.109411 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a3c87d-b0e5-464b-a910-7c5ec81517dd None None] PATH_INFO: `/` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:58.111952 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 484/977] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 26 10:52:58 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 26 10:52:58.116872 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2859f4ed-575a-4953-b72a-e7915ebbf0e2 None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:58.117151 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2859f4ed-575a-4953-b72a-e7915ebbf0e2 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:58.117422 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2859f4ed-575a-4953-b72a-e7915ebbf0e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:58.137077 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-2859f4ed-575a-4953-b72a-e7915ebbf0e2 None None] MFA Rules not processed for user `ca155a48e2024c32af3fd2e19e5fac11`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 10:52:58.139412 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-2859f4ed-575a-4953-b72a-e7915ebbf0e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 10:52:58.146734 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:52:58.171521 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12880d38-d5b1-4fd8-85eb-677dd644b18a None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:52:58.210075 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60136fc-ea45-4ab7-9094-674421b8b978 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:52:58.210657 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60136fc-ea45-4ab7-9094-674421b8b978 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:58.210929 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60136fc-ea45-4ab7-9094-674421b8b978 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:58.211198 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60136fc-ea45-4ab7-9094-674421b8b978 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:58.223075 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 485/978] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 26 10:52:58 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 26 10:52:58.223228 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b60136fc-ea45-4ab7-9094-674421b8b978 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:52:58.224746 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b60136fc-ea45-4ab7-9094-674421b8b978 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:52:58.241270 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application [None req-b60136fc-ea45-4ab7-9094-674421b8b978 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 10:52:58.241270 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 10:52:58.241270 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 10:52:58.241270 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 10:52:58.241270 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 10:52:58.241270 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 10:52:58.241270 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 10:52:58.241270 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 10:52:58.241270 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 10:52:58.241270 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 10:52:58.241270 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 10:52:58.241270 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 10:52:58.241270 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 10:52:58.241270 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 10:52:58.241270 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 10:52:58.241270 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 10:52:58.242465 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 10:52:58.242465 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 10:52:58.242465 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 10:52:58.242465 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 10:52:58.242465 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 10:52:58.242465 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 10:52:58.242465 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 10:52:58.242465 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application with Lock( Aug 26 10:52:58.242465 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 10:52:58.242465 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._enter() Aug 26 10:52:58.242465 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 10:52:58.242465 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 10:52:58.242465 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 10:52:58.242465 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.creator() Aug 26 10:52:58.242465 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 10:52:58.242465 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application created_value = creator( Aug 26 10:52:58.242465 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 10:52:58.243573 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 10:52:58.243573 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 10:52:58.243573 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Aug 26 10:52:58.248677 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 494/979] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Aug 26 10:52:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 10:52:58.249742 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2859f4ed-575a-4953-b72a-e7915ebbf0e2 None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:52:58.351908 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2859f4ed-575a-4953-b72a-e7915ebbf0e2 None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:52:58.355324 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60136fc-ea45-4ab7-9094-674421b8b978 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:52:58.355735 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:52:58.390592 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4017a0d-db12-447e-a954-70578f49d5ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:52:58.391894 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4017a0d-db12-447e-a954-70578f49d5ee None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:58.392161 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4017a0d-db12-447e-a954-70578f49d5ee None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:58.392442 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4017a0d-db12-447e-a954-70578f49d5ee None admin] PATH_INFO: `/v3/domains` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:58.393850 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4017a0d-db12-447e-a954-70578f49d5ee None admin] RBAC: Adding query filter params (name=Default) {{(pid=81423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 10:52:58.394184 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4017a0d-db12-447e-a954-70578f49d5ee None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:52:58.395558 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4017a0d-db12-447e-a954-70578f49d5ee None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:52:58.407701 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 486/980] 10.222.0.40 () {56 vars in 1109 bytes} [Fri Aug 26 10:52:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 10:52:58.413239 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4017a0d-db12-447e-a954-70578f49d5ee None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:52:58.463543 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60136fc-ea45-4ab7-9094-674421b8b978 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:52:58.492956 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:52:58.499999 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b6df4c-f1ec-484f-9281-3cc4deecf11b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:52:58.500888 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b6df4c-f1ec-484f-9281-3cc4deecf11b None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:58.500888 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b6df4c-f1ec-484f-9281-3cc4deecf11b None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:58.501281 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b6df4c-f1ec-484f-9281-3cc4deecf11b None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:58.501864 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b6df4c-f1ec-484f-9281-3cc4deecf11b None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:52:58.503244 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b6df4c-f1ec-484f-9281-3cc4deecf11b None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:52:58.511931 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application [None req-65b6df4c-f1ec-484f-9281-3cc4deecf11b None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 26 10:52:58.511931 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 10:52:58.511931 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 26 10:52:58.511931 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 26 10:52:58.511931 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 26 10:52:58.511931 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 26 10:52:58.511931 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 26 10:52:58.511931 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 26 10:52:58.511931 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 26 10:52:58.511931 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Aug 26 10:52:58.511931 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 26 10:52:58.511931 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Aug 26 10:52:58.511931 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 10:52:58.511931 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 10:52:58.511931 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 10:52:58.511931 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 10:52:58.511931 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 10:52:58.513059 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 10:52:58.513059 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 10:52:58.513059 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 10:52:58.513059 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 10:52:58.513059 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 10:52:58.513059 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 10:52:58.513059 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 26 10:52:58.513059 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 26 10:52:58.513059 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 26 10:52:58.513059 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 26 10:52:58.513059 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 10:52:58.513059 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 10:52:58.513059 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 10:52:58.513059 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 10:52:58.513059 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 10:52:58.513059 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 10:52:58.514374 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 10:52:58.514374 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application with Lock( Aug 26 10:52:58.514374 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 10:52:58.514374 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._enter() Aug 26 10:52:58.514374 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 10:52:58.514374 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 10:52:58.514374 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 10:52:58.514374 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.creator() Aug 26 10:52:58.514374 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 10:52:58.514374 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application created_value = creator( Aug 26 10:52:58.514374 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 26 10:52:58.514374 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 26 10:52:58.514374 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 26 10:52:58.514374 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Aug 26 10:52:58.514374 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 495/981] 10.222.0.40 () {56 vars in 1126 bytes} [Fri Aug 26 10:52:58 2022] GET /identity/v3/services/object-store => generated 93 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 26 10:52:58.515450 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b6df4c-f1ec-484f-9281-3cc4deecf11b None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:52:58.534968 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4017a0d-db12-447e-a954-70578f49d5ee None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:52:58.570059 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e41fbf5-f2da-42e4-9c71-a3efb011e06c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:52:58.570681 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e41fbf5-f2da-42e4-9c71-a3efb011e06c None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:58.570958 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e41fbf5-f2da-42e4-9c71-a3efb011e06c None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:58.571226 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e41fbf5-f2da-42e4-9c71-a3efb011e06c None admin] PATH_INFO: `/v3/projects/service` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:58.577738 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e41fbf5-f2da-42e4-9c71-a3efb011e06c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:52:58.579757 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e41fbf5-f2da-42e4-9c71-a3efb011e06c None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:52:58.592224 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application [None req-5e41fbf5-f2da-42e4-9c71-a3efb011e06c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 10:52:58.592224 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 10:52:58.592224 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 10:52:58.592224 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 10:52:58.592224 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 10:52:58.592224 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 10:52:58.592224 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 10:52:58.592224 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 10:52:58.592224 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 10:52:58.592224 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 10:52:58.592224 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 10:52:58.592224 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 10:52:58.592224 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 26 10:52:58.592224 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 26 10:52:58.592224 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 26 10:52:58.593321 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 26 10:52:58.593321 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 10:52:58.593321 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 10:52:58.593321 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 10:52:58.593321 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 10:52:58.593321 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 10:52:58.593321 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 10:52:58.593321 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 10:52:58.593321 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application with Lock( Aug 26 10:52:58.593321 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 10:52:58.593321 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._enter() Aug 26 10:52:58.593321 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 10:52:58.593321 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 10:52:58.593321 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 10:52:58.593321 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.creator() Aug 26 10:52:58.593321 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 10:52:58.593321 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application created_value = creator( Aug 26 10:52:58.594434 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 26 10:52:58.594434 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 26 10:52:58.594434 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 26 10:52:58.594434 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 26 10:52:58.594434 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 26 10:52:58.594434 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 26 10:52:58.594434 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 10:52:58.594434 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Aug 26 10:52:58.594434 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 487/982] 10.222.0.40 () {56 vars in 1111 bytes} [Fri Aug 26 10:52:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 26 10:52:58.595049 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e41fbf5-f2da-42e4-9c71-a3efb011e06c None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:52:58.619628 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b6df4c-f1ec-484f-9281-3cc4deecf11b None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:52:58.620072 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:52:58.654065 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f463c58-c4f4-4455-b91a-30489378a404 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:52:58.654652 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f463c58-c4f4-4455-b91a-30489378a404 None admin] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:58.654932 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f463c58-c4f4-4455-b91a-30489378a404 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:58.655259 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f463c58-c4f4-4455-b91a-30489378a404 None admin] PATH_INFO: `/v3/services` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:58.655923 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f463c58-c4f4-4455-b91a-30489378a404 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:52:58.657288 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f463c58-c4f4-4455-b91a-30489378a404 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:52:58.671140 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 496/983] 10.222.0.40 () {60 vars in 1141 bytes} [Fri Aug 26 10:52:58 2022] POST /identity/v3/services => generated 242 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 10:52:58.672075 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f463c58-c4f4-4455-b91a-30489378a404 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:52:58.706643 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e41fbf5-f2da-42e4-9c71-a3efb011e06c None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:52:58.733066 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:52:58.755619 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a5b66b-a7ec-49c9-b57e-f7350ecfbe53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:52:58.756235 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a5b66b-a7ec-49c9-b57e-f7350ecfbe53 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:58.756514 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a5b66b-a7ec-49c9-b57e-f7350ecfbe53 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:58.756781 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a5b66b-a7ec-49c9-b57e-f7350ecfbe53 None admin] PATH_INFO: `/v3/services` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:58.757422 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a5b66b-a7ec-49c9-b57e-f7350ecfbe53 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 10:52:58.757899 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a5b66b-a7ec-49c9-b57e-f7350ecfbe53 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:52:58.759368 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a5b66b-a7ec-49c9-b57e-f7350ecfbe53 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:52:58.767070 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 488/984] 10.222.0.40 () {56 vars in 1122 bytes} [Fri Aug 26 10:52:58 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 10:52:58.772689 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a5b66b-a7ec-49c9-b57e-f7350ecfbe53 None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:52:58.792037 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f463c58-c4f4-4455-b91a-30489378a404 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:52:58.835151 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a65d061-5552-42db-a24b-c23be2202927 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:52:58.835761 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a65d061-5552-42db-a24b-c23be2202927 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:58.836113 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a65d061-5552-42db-a24b-c23be2202927 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:58.836290 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a65d061-5552-42db-a24b-c23be2202927 None admin] PATH_INFO: `/v3/projects` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:58.836975 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a65d061-5552-42db-a24b-c23be2202927 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 10:52:58.837370 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a65d061-5552-42db-a24b-c23be2202927 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:52:58.839889 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a65d061-5552-42db-a24b-c23be2202927 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:52:58.858823 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 497/985] 10.222.0.40 () {56 vars in 1148 bytes} [Fri Aug 26 10:52:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 10:52:58.865579 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a65d061-5552-42db-a24b-c23be2202927 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:52:58.868997 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:52:58.893372 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a5b66b-a7ec-49c9-b57e-f7350ecfbe53 None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:52:58.931974 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f0f56e-807b-4517-be15-8f893435f58e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:52:58.932576 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f0f56e-807b-4517-be15-8f893435f58e None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:58.932814 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f0f56e-807b-4517-be15-8f893435f58e None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:58.933051 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f0f56e-807b-4517-be15-8f893435f58e None admin] PATH_INFO: `/v3/services` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:58.933770 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f0f56e-807b-4517-be15-8f893435f58e None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 10:52:58.934027 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f0f56e-807b-4517-be15-8f893435f58e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:52:58.935551 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f0f56e-807b-4517-be15-8f893435f58e None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:52:58.941741 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 489/986] 10.222.0.40 () {56 vars in 1122 bytes} [Fri Aug 26 10:52:58 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 10:52:58.947661 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f0f56e-807b-4517-be15-8f893435f58e None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:52:58.971347 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a65d061-5552-42db-a24b-c23be2202927 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:52:58.987521 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:52:59.009585 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016a17e9-4547-4fe1-b907-20e0b45b58ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:52:59.010225 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016a17e9-4547-4fe1-b907-20e0b45b58ce None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:59.010562 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016a17e9-4547-4fe1-b907-20e0b45b58ce None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:59.010844 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016a17e9-4547-4fe1-b907-20e0b45b58ce None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:59.011563 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-016a17e9-4547-4fe1-b907-20e0b45b58ce None admin] RBAC: Adding query filter params (role.id=3f7d20781437466aabcbf11bda248f3d, scope.project.id=0e30ccf0c1204727b786876c141dbcd2, user.id=50be12ccbd274a8792615e9dd10b61ca) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 10:52:59.011868 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-016a17e9-4547-4fe1-b907-20e0b45b58ce None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:52:59.013294 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-016a17e9-4547-4fe1-b907-20e0b45b58ce None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:52:59.030022 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 498/987] 10.222.0.40 () {56 vars in 1374 bytes} [Fri Aug 26 10:52:58 2022] GET /identity/v3/role_assignments?user.id=50be12ccbd274a8792615e9dd10b61ca&scope.project.id=0e30ccf0c1204727b786876c141dbcd2&role.id=3f7d20781437466aabcbf11bda248f3d => generated 608 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 10:52:59.070393 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f0f56e-807b-4517-be15-8f893435f58e None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:52:59.109141 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f595899-dbd8-4c7f-a9e9-a5b8d9180c2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:52:59.109882 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f595899-dbd8-4c7f-a9e9-a5b8d9180c2b None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:59.110189 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f595899-dbd8-4c7f-a9e9-a5b8d9180c2b None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:59.110462 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f595899-dbd8-4c7f-a9e9-a5b8d9180c2b None admin] PATH_INFO: `/v3/endpoints` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:59.111184 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f595899-dbd8-4c7f-a9e9-a5b8d9180c2b None admin] RBAC: Adding query filter params (interface=admin, service_id=1ad4e0d9c1544d42b1d4a571a6262d4e, region_id=RegionOne) {{(pid=81423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 10:52:59.111491 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f595899-dbd8-4c7f-a9e9-a5b8d9180c2b None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:52:59.112872 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f595899-dbd8-4c7f-a9e9-a5b8d9180c2b None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:52:59.118743 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 490/988] 10.222.0.40 () {56 vars in 1249 bytes} [Fri Aug 26 10:52:58 2022] GET /identity/v3/endpoints?service_id=1ad4e0d9c1544d42b1d4a571a6262d4e&interface=admin®ion_id=RegionOne => generated 193 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 10:52:59.124611 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016a17e9-4547-4fe1-b907-20e0b45b58ce None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:52:59.236594 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016a17e9-4547-4fe1-b907-20e0b45b58ce None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:52:59.271605 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ef862f-654b-486f-b933-cb37eb955cf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:52:59.272213 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ef862f-654b-486f-b933-cb37eb955cf4 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:52:59.272491 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ef862f-654b-486f-b933-cb37eb955cf4 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:52:59.272763 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ef862f-654b-486f-b933-cb37eb955cf4 None admin] PATH_INFO: `/v3/services` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:52:59.273369 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ef862f-654b-486f-b933-cb37eb955cf4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:52:59.274761 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ef862f-654b-486f-b933-cb37eb955cf4 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:52:59.280988 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 499/989] 10.222.0.40 () {56 vars in 1087 bytes} [Fri Aug 26 10:52:59 2022] GET /identity/v3/services => generated 1727 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 10:53:00.618237 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b07516-f536-423a-82c0-50944d26836d None None] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:53:00.618797 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b07516-f536-423a-82c0-50944d26836d None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:53:00.618797 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b07516-f536-423a-82c0-50944d26836d None None] PATH_INFO: `/` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:53:00.620271 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 491/990] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 26 10:53:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 10:53:00.626008 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6a2266-f78e-42be-943c-d10c53166053 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:53:00.626423 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6a2266-f78e-42be-943c-d10c53166053 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:53:00.626587 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6a2266-f78e-42be-943c-d10c53166053 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:53:00.645759 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-1e6a2266-f78e-42be-943c-d10c53166053 None None] MFA Rules not processed for user `ca155a48e2024c32af3fd2e19e5fac11`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 10:53:00.646982 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-1e6a2266-f78e-42be-943c-d10c53166053 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 10:53:00.682455 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:53:00.718068 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 500/991] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 26 10:53:00 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 26 10:53:00.988975 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa57bb2-87ec-4d56-8c2f-b509bc81ae01 None None] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:53:00.989354 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa57bb2-87ec-4d56-8c2f-b509bc81ae01 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:53:00.989547 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa57bb2-87ec-4d56-8c2f-b509bc81ae01 None None] PATH_INFO: `/` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:53:00.989656 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6a2266-f78e-42be-943c-d10c53166053 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:53:00.991024 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 492/992] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 26 10:53:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 10:53:00.995794 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39bc9cd-d705-4dcc-a7e0-d41368c84fe1 None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:53:00.996072 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39bc9cd-d705-4dcc-a7e0-d41368c84fe1 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:53:00.996343 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39bc9cd-d705-4dcc-a7e0-d41368c84fe1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:53:01.014091 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-a39bc9cd-d705-4dcc-a7e0-d41368c84fe1 None None] MFA Rules not processed for user `ca155a48e2024c32af3fd2e19e5fac11`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 10:53:01.015280 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-a39bc9cd-d705-4dcc-a7e0-d41368c84fe1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 10:53:01.022739 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:53:01.084869 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 493/993] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 26 10:53:00 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 26 10:53:01.096750 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6a2266-f78e-42be-943c-d10c53166053 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:53:01.131653 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0f4585-29e0-47b0-991f-87d164864ce7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:53:01.132550 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0f4585-29e0-47b0-991f-87d164864ce7 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:53:01.133009 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0f4585-29e0-47b0-991f-87d164864ce7 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:53:01.133479 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0f4585-29e0-47b0-991f-87d164864ce7 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:53:01.134278 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca0f4585-29e0-47b0-991f-87d164864ce7 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:53:01.135812 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca0f4585-29e0-47b0-991f-87d164864ce7 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:53:01.145255 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application [None req-ca0f4585-29e0-47b0-991f-87d164864ce7 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 26 10:53:01.145255 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 10:53:01.145255 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 26 10:53:01.145255 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 26 10:53:01.145255 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 26 10:53:01.145255 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 26 10:53:01.145255 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 26 10:53:01.145255 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 26 10:53:01.145255 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 26 10:53:01.145255 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Aug 26 10:53:01.145255 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 26 10:53:01.145255 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Aug 26 10:53:01.145255 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 10:53:01.145255 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 10:53:01.145255 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 10:53:01.145255 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 10:53:01.145255 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 10:53:01.146442 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 10:53:01.146442 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 10:53:01.146442 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 10:53:01.146442 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 10:53:01.146442 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 10:53:01.146442 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 10:53:01.146442 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 26 10:53:01.146442 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 26 10:53:01.146442 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 26 10:53:01.146442 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 26 10:53:01.146442 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 10:53:01.146442 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 10:53:01.146442 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 10:53:01.146442 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 10:53:01.146442 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 10:53:01.146442 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 10:53:01.147509 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 10:53:01.147509 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application with Lock( Aug 26 10:53:01.147509 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 10:53:01.147509 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._enter() Aug 26 10:53:01.147509 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 10:53:01.147509 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 10:53:01.147509 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 10:53:01.147509 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.creator() Aug 26 10:53:01.147509 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 10:53:01.147509 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application created_value = creator( Aug 26 10:53:01.147509 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 26 10:53:01.147509 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 26 10:53:01.147509 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 26 10:53:01.147509 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Aug 26 10:53:01.149575 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 501/994] 10.222.0.40 () {56 vars in 1111 bytes} [Fri Aug 26 10:53:00 2022] GET /identity/v3/services/compute => generated 88 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 10:53:01.155414 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39bc9cd-d705-4dcc-a7e0-d41368c84fe1 None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:53:01.241083 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7946695f-9a4d-4996-a905-0480e74040a5 None None] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:53:01.241628 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7946695f-9a4d-4996-a905-0480e74040a5 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:53:01.242194 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7946695f-9a4d-4996-a905-0480e74040a5 None None] PATH_INFO: `/` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:53:01.243685 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 502/995] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 26 10:53:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 26 10:53:01.248974 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba0a6ca-ac7d-4ce1-9b9b-5b16ccbb651d None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:53:01.249464 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba0a6ca-ac7d-4ce1-9b9b-5b16ccbb651d None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:53:01.249916 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba0a6ca-ac7d-4ce1-9b9b-5b16ccbb651d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:53:01.263639 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39bc9cd-d705-4dcc-a7e0-d41368c84fe1 None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:53:01.269950 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-6ba0a6ca-ac7d-4ce1-9b9b-5b16ccbb651d None None] MFA Rules not processed for user `ca155a48e2024c32af3fd2e19e5fac11`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 10:53:01.271364 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-6ba0a6ca-ac7d-4ce1-9b9b-5b16ccbb651d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 10:53:01.300416 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a37d31-b2cd-40fd-ba7d-3a21e361d692 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:53:01.301198 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a37d31-b2cd-40fd-ba7d-3a21e361d692 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:53:01.301333 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a37d31-b2cd-40fd-ba7d-3a21e361d692 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:53:01.301559 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a37d31-b2cd-40fd-ba7d-3a21e361d692 None admin] PATH_INFO: `/v3/services` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:53:01.302280 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a37d31-b2cd-40fd-ba7d-3a21e361d692 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 10:53:01.302541 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a37d31-b2cd-40fd-ba7d-3a21e361d692 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:53:01.303858 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a37d31-b2cd-40fd-ba7d-3a21e361d692 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:53:01.304099 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:53:01.314118 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 494/996] 10.222.0.40 () {56 vars in 1112 bytes} [Fri Aug 26 10:53:01 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 10:53:01.314981 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a37d31-b2cd-40fd-ba7d-3a21e361d692 None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:53:01.323623 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:53:01.345843 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 503/997] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 26 10:53:01 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 26 10:53:01.361492 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba0a6ca-ac7d-4ce1-9b9b-5b16ccbb651d None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:53:01.427359 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a37d31-b2cd-40fd-ba7d-3a21e361d692 None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:53:01.464403 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc21f9bf-efa2-4bcf-ac8a-1f04a4fc9b20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:53:01.465283 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc21f9bf-efa2-4bcf-ac8a-1f04a4fc9b20 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:53:01.465686 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba0a6ca-ac7d-4ce1-9b9b-5b16ccbb651d None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:53:01.465966 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc21f9bf-efa2-4bcf-ac8a-1f04a4fc9b20 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:53:01.466476 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc21f9bf-efa2-4bcf-ac8a-1f04a4fc9b20 None admin] PATH_INFO: `/v3/services` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:53:01.467282 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc21f9bf-efa2-4bcf-ac8a-1f04a4fc9b20 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 10:53:01.467770 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc21f9bf-efa2-4bcf-ac8a-1f04a4fc9b20 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:53:01.469869 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc21f9bf-efa2-4bcf-ac8a-1f04a4fc9b20 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:53:01.475422 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 495/998] 10.222.0.40 () {56 vars in 1112 bytes} [Fri Aug 26 10:53:01 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 10:53:01.480818 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc21f9bf-efa2-4bcf-ac8a-1f04a4fc9b20 None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:53:01.506711 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd13598-637f-42b2-b046-d7ba2bb715c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:53:01.507351 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd13598-637f-42b2-b046-d7ba2bb715c1 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:53:01.507630 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd13598-637f-42b2-b046-d7ba2bb715c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:53:01.507904 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd13598-637f-42b2-b046-d7ba2bb715c1 None admin] PATH_INFO: `/v3/services/image` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:53:01.508486 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd13598-637f-42b2-b046-d7ba2bb715c1 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:53:01.509845 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd13598-637f-42b2-b046-d7ba2bb715c1 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:53:01.520369 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application [None req-9fd13598-637f-42b2-b046-d7ba2bb715c1 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 26 10:53:01.520369 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 10:53:01.520369 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 26 10:53:01.520369 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 26 10:53:01.520369 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 26 10:53:01.520369 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 26 10:53:01.520369 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 26 10:53:01.520369 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 26 10:53:01.520369 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 26 10:53:01.520369 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Aug 26 10:53:01.520369 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 26 10:53:01.520369 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Aug 26 10:53:01.520369 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 10:53:01.520369 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 10:53:01.520369 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 10:53:01.520369 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 10:53:01.520369 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 10:53:01.522214 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 10:53:01.522214 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 10:53:01.522214 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 10:53:01.522214 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 10:53:01.522214 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 10:53:01.522214 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 10:53:01.522214 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 26 10:53:01.522214 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 26 10:53:01.522214 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 26 10:53:01.522214 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 26 10:53:01.522214 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 10:53:01.522214 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 10:53:01.522214 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 10:53:01.522214 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 10:53:01.522214 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 10:53:01.522214 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 10:53:01.523129 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 10:53:01.523129 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application with Lock( Aug 26 10:53:01.523129 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 10:53:01.523129 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._enter() Aug 26 10:53:01.523129 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 10:53:01.523129 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 10:53:01.523129 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 10:53:01.523129 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.creator() Aug 26 10:53:01.523129 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 10:53:01.523129 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application created_value = creator( Aug 26 10:53:01.523129 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 26 10:53:01.523129 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 26 10:53:01.523129 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 26 10:53:01.523129 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Aug 26 10:53:01.523129 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 504/999] 10.222.0.40 () {56 vars in 1105 bytes} [Fri Aug 26 10:53:01 2022] GET /identity/v3/services/image => generated 86 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 26 10:53:01.527230 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd13598-637f-42b2-b046-d7ba2bb715c1 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:53:01.558455 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:53:01.558788 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:53:01.591643 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc21f9bf-efa2-4bcf-ac8a-1f04a4fc9b20 None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:53:01.626628 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5dfb5a0-8ad7-4642-a8f6-d0e5fa0667ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:53:01.627280 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5dfb5a0-8ad7-4642-a8f6-d0e5fa0667ea None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:53:01.627560 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5dfb5a0-8ad7-4642-a8f6-d0e5fa0667ea None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:53:01.627862 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5dfb5a0-8ad7-4642-a8f6-d0e5fa0667ea None admin] PATH_INFO: `/v3/endpoints` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:53:01.628560 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5dfb5a0-8ad7-4642-a8f6-d0e5fa0667ea None admin] RBAC: Adding query filter params (interface=public, service_id=764833969ca641f787c8e2866edcaccf, region_id=RegionOne) {{(pid=81423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 10:53:01.628863 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5dfb5a0-8ad7-4642-a8f6-d0e5fa0667ea None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:53:01.630413 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5dfb5a0-8ad7-4642-a8f6-d0e5fa0667ea None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:53:01.634372 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd13598-637f-42b2-b046-d7ba2bb715c1 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:53:01.635244 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 496/1000] 10.222.0.40 () {56 vars in 1251 bytes} [Fri Aug 26 10:53:01 2022] GET /identity/v3/endpoints?service_id=764833969ca641f787c8e2866edcaccf&interface=public®ion_id=RegionOne => generated 194 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 10:53:01.636005 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5dfb5a0-8ad7-4642-a8f6-d0e5fa0667ea None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:53:01.669703 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c0cc5f-2262-4098-9044-b7b910f3df80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:53:01.670342 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c0cc5f-2262-4098-9044-b7b910f3df80 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:53:01.670615 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c0cc5f-2262-4098-9044-b7b910f3df80 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:53:01.670879 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c0cc5f-2262-4098-9044-b7b910f3df80 None admin] PATH_INFO: `/v3/services` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:53:01.671492 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c0cc5f-2262-4098-9044-b7b910f3df80 None admin] RBAC: Adding query filter params (name=image) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 10:53:01.671787 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c0cc5f-2262-4098-9044-b7b910f3df80 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:53:01.673229 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c0cc5f-2262-4098-9044-b7b910f3df80 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:53:01.677966 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 505/1001] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Aug 26 10:53:01 2022] GET /identity/v3/services?name=image => generated 122 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 10:53:01.678725 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c0cc5f-2262-4098-9044-b7b910f3df80 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:53:01.741992 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5dfb5a0-8ad7-4642-a8f6-d0e5fa0667ea None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:53:01.770544 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:53:01.775961 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4cb834d-9e66-4530-8374-2042e8fb25a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:53:01.776591 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cb834d-9e66-4530-8374-2042e8fb25a8 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:53:01.776870 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cb834d-9e66-4530-8374-2042e8fb25a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:53:01.777140 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cb834d-9e66-4530-8374-2042e8fb25a8 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:53:01.777573 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c0cc5f-2262-4098-9044-b7b910f3df80 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:53:01.777720 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4cb834d-9e66-4530-8374-2042e8fb25a8 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:53:01.779106 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4cb834d-9e66-4530-8374-2042e8fb25a8 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:53:01.780829 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:53:01.788764 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application [None req-d4cb834d-9e66-4530-8374-2042e8fb25a8 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 26 10:53:01.788764 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 10:53:01.788764 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 26 10:53:01.788764 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 26 10:53:01.788764 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 26 10:53:01.788764 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 26 10:53:01.788764 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 26 10:53:01.788764 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 26 10:53:01.788764 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 26 10:53:01.788764 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Aug 26 10:53:01.788764 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 26 10:53:01.788764 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Aug 26 10:53:01.788764 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 10:53:01.788764 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 10:53:01.788764 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 10:53:01.788764 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 10:53:01.788764 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 10:53:01.790002 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 10:53:01.790002 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 10:53:01.790002 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 10:53:01.790002 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 10:53:01.790002 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 10:53:01.790002 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 10:53:01.790002 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 26 10:53:01.790002 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 26 10:53:01.790002 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 26 10:53:01.790002 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 26 10:53:01.790002 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 10:53:01.790002 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 10:53:01.790002 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 10:53:01.790002 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 10:53:01.790002 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 10:53:01.790002 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 10:53:01.790887 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 10:53:01.790887 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application with Lock( Aug 26 10:53:01.790887 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 10:53:01.790887 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._enter() Aug 26 10:53:01.790887 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 10:53:01.790887 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 10:53:01.790887 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 10:53:01.790887 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.creator() Aug 26 10:53:01.790887 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 10:53:01.790887 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application created_value = creator( Aug 26 10:53:01.790887 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 26 10:53:01.790887 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 26 10:53:01.790887 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 26 10:53:01.790887 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Aug 26 10:53:01.790887 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 497/1002] 10.222.0.40 () {56 vars in 1126 bytes} [Fri Aug 26 10:53:01 2022] GET /identity/v3/services/object-store => generated 93 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 10:53:01.791713 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4cb834d-9e66-4530-8374-2042e8fb25a8 None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:53:01.815823 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a4d7b5-9ede-4ccc-8ceb-6f9feb251465 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:53:01.816421 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a4d7b5-9ede-4ccc-8ceb-6f9feb251465 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:53:01.816705 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a4d7b5-9ede-4ccc-8ceb-6f9feb251465 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:53:01.816977 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a4d7b5-9ede-4ccc-8ceb-6f9feb251465 None admin] PATH_INFO: `/v3/services` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:53:01.817578 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a4d7b5-9ede-4ccc-8ceb-6f9feb251465 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:53:01.818973 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a4d7b5-9ede-4ccc-8ceb-6f9feb251465 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:53:01.825011 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 506/1003] 10.222.0.40 () {56 vars in 1087 bytes} [Fri Aug 26 10:53:01 2022] GET /identity/v3/services => generated 1727 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 10:53:01.825785 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a4d7b5-9ede-4ccc-8ceb-6f9feb251465 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:53:01.913312 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4cb834d-9e66-4530-8374-2042e8fb25a8 None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:53:01.949178 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a4d7b5-9ede-4ccc-8ceb-6f9feb251465 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:53:01.953077 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfebced4-0677-4d30-bab8-7cfd3ac3e10a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:53:01.953737 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfebced4-0677-4d30-bab8-7cfd3ac3e10a None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:53:01.954037 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfebced4-0677-4d30-bab8-7cfd3ac3e10a None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:53:01.954403 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfebced4-0677-4d30-bab8-7cfd3ac3e10a None admin] PATH_INFO: `/v3/services` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:53:01.955053 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfebced4-0677-4d30-bab8-7cfd3ac3e10a None admin] RBAC: Adding query filter params (type=image) {{(pid=81423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 10:53:01.955356 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfebced4-0677-4d30-bab8-7cfd3ac3e10a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:53:01.956746 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfebced4-0677-4d30-bab8-7cfd3ac3e10a None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:53:01.961825 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 498/1004] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Aug 26 10:53:01 2022] GET /identity/v3/services?type=image => generated 122 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 10:53:01.997634 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d539062-aae9-4330-bcf0-fc4f3ccfd92f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:53:01.998291 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d539062-aae9-4330-bcf0-fc4f3ccfd92f None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:53:01.998571 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d539062-aae9-4330-bcf0-fc4f3ccfd92f None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:53:01.999286 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d539062-aae9-4330-bcf0-fc4f3ccfd92f None admin] PATH_INFO: `/v3/services` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:53:01.999476 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d539062-aae9-4330-bcf0-fc4f3ccfd92f None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 10:53:01.999778 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d539062-aae9-4330-bcf0-fc4f3ccfd92f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:53:02.001768 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d539062-aae9-4330-bcf0-fc4f3ccfd92f None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:53:02.007364 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 507/1005] 10.222.0.40 () {56 vars in 1122 bytes} [Fri Aug 26 10:53:01 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 10:53:02.013204 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfebced4-0677-4d30-bab8-7cfd3ac3e10a None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:53:02.080977 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:53:02.127808 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfebced4-0677-4d30-bab8-7cfd3ac3e10a None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:53:02.163265 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5bf72d7-6d5b-4917-aed4-70367bc0f378 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:53:02.163875 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bf72d7-6d5b-4917-aed4-70367bc0f378 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:53:02.164149 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bf72d7-6d5b-4917-aed4-70367bc0f378 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:53:02.164416 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bf72d7-6d5b-4917-aed4-70367bc0f378 None admin] PATH_INFO: `/v3/services` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:53:02.165042 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5bf72d7-6d5b-4917-aed4-70367bc0f378 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 10:53:02.165340 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5bf72d7-6d5b-4917-aed4-70367bc0f378 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:53:02.166721 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5bf72d7-6d5b-4917-aed4-70367bc0f378 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:53:02.171765 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 499/1006] 10.222.0.40 () {56 vars in 1122 bytes} [Fri Aug 26 10:53:02 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 10:53:02.178507 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d539062-aae9-4330-bcf0-fc4f3ccfd92f None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:53:02.204674 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:53:02.287645 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d539062-aae9-4330-bcf0-fc4f3ccfd92f None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:53:02.322895 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d9cf71-2854-4710-83c9-baac10fee422 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:53:02.323510 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d9cf71-2854-4710-83c9-baac10fee422 None admin] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:53:02.323791 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d9cf71-2854-4710-83c9-baac10fee422 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:53:02.324080 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d9cf71-2854-4710-83c9-baac10fee422 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:53:02.324720 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d9cf71-2854-4710-83c9-baac10fee422 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:53:02.326118 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d9cf71-2854-4710-83c9-baac10fee422 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:53:02.342781 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 508/1007] 10.222.0.40 () {60 vars in 1144 bytes} [Fri Aug 26 10:53:02 2022] POST /identity/v3/endpoints => generated 325 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 10:53:03.774718 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e76bd3d-b00f-4ca3-8a83-8102387abed7 None None] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:53:03.775239 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e76bd3d-b00f-4ca3-8a83-8102387abed7 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:53:03.775239 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e76bd3d-b00f-4ca3-8a83-8102387abed7 None None] PATH_INFO: `/` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:53:03.776729 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 500/1008] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 26 10:53:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 10:53:03.782792 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bd968e-7dd4-4928-a7e4-b83bd4933b9a None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:53:03.783182 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bd968e-7dd4-4928-a7e4-b83bd4933b9a None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:53:03.783345 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bd968e-7dd4-4928-a7e4-b83bd4933b9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:53:03.801072 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-71bd968e-7dd4-4928-a7e4-b83bd4933b9a None None] MFA Rules not processed for user `ca155a48e2024c32af3fd2e19e5fac11`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 10:53:03.802276 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-71bd968e-7dd4-4928-a7e4-b83bd4933b9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 10:53:03.879253 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bd6420-947e-4276-a3c8-e0a0ac9be247 None None] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:53:03.879603 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bd6420-947e-4276-a3c8-e0a0ac9be247 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:53:03.879939 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bd6420-947e-4276-a3c8-e0a0ac9be247 None None] PATH_INFO: `/` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:53:03.881355 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 501/1009] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 26 10:53:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 10:53:03.884213 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 509/1010] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 26 10:53:03 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 26 10:53:03.886477 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca59ba3-62fa-481a-8d7e-7232c6f973db None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:53:03.886762 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca59ba3-62fa-481a-8d7e-7232c6f973db None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:53:03.887040 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca59ba3-62fa-481a-8d7e-7232c6f973db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:53:03.905566 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-7ca59ba3-62fa-481a-8d7e-7232c6f973db None None] MFA Rules not processed for user `ca155a48e2024c32af3fd2e19e5fac11`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 10:53:03.906817 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-7ca59ba3-62fa-481a-8d7e-7232c6f973db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 10:53:03.976206 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 502/1011] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 26 10:53:03 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 26 10:53:04.159244 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71bd968e-7dd4-4928-a7e4-b83bd4933b9a None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:53:04.170545 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:53:04.247896 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca59ba3-62fa-481a-8d7e-7232c6f973db None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:53:04.270510 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71bd968e-7dd4-4928-a7e4-b83bd4933b9a None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:53:04.306301 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:53:04.310874 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a538fc6-561a-479d-9a7a-7d67f005392c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:53:04.311508 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a538fc6-561a-479d-9a7a-7d67f005392c None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:53:04.311782 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a538fc6-561a-479d-9a7a-7d67f005392c None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:53:04.312049 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a538fc6-561a-479d-9a7a-7d67f005392c None admin] PATH_INFO: `/v3/services/compute` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:53:04.312632 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a538fc6-561a-479d-9a7a-7d67f005392c None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:53:04.314090 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a538fc6-561a-479d-9a7a-7d67f005392c None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:53:04.322906 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application [None req-3a538fc6-561a-479d-9a7a-7d67f005392c None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 26 10:53:04.322906 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 10:53:04.322906 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 26 10:53:04.322906 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 26 10:53:04.322906 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 26 10:53:04.322906 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 26 10:53:04.322906 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 26 10:53:04.322906 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 26 10:53:04.322906 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 26 10:53:04.322906 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Aug 26 10:53:04.322906 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 26 10:53:04.322906 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Aug 26 10:53:04.322906 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 10:53:04.322906 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 10:53:04.322906 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 10:53:04.322906 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 10:53:04.322906 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 10:53:04.323905 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 10:53:04.323905 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 10:53:04.323905 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 10:53:04.323905 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 10:53:04.323905 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 10:53:04.323905 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 10:53:04.323905 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 26 10:53:04.323905 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 26 10:53:04.323905 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 26 10:53:04.323905 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 26 10:53:04.323905 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 10:53:04.323905 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 10:53:04.323905 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 10:53:04.323905 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 10:53:04.323905 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 10:53:04.323905 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 10:53:04.324744 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 10:53:04.324744 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application with Lock( Aug 26 10:53:04.324744 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 10:53:04.324744 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._enter() Aug 26 10:53:04.324744 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 10:53:04.324744 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 10:53:04.324744 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 10:53:04.324744 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.creator() Aug 26 10:53:04.324744 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 10:53:04.324744 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application created_value = creator( Aug 26 10:53:04.324744 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 26 10:53:04.324744 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 26 10:53:04.324744 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 26 10:53:04.324744 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Aug 26 10:53:04.324744 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 510/1012] 10.222.0.40 () {56 vars in 1111 bytes} [Fri Aug 26 10:53:04 2022] GET /identity/v3/services/compute => generated 88 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 10:53:04.326072 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8216e908-0846-421e-9621-8ac7db9bd79f None None] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:53:04.326072 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8216e908-0846-421e-9621-8ac7db9bd79f None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:53:04.326072 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8216e908-0846-421e-9621-8ac7db9bd79f None None] PATH_INFO: `/` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:53:04.327118 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 511/1013] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 26 10:53:04 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 10:53:04.331015 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8216e908-0846-421e-9621-8ac7db9bd79f None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:53:04.351785 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca59ba3-62fa-481a-8d7e-7232c6f973db None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:53:04.387173 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98cb5e95-1411-4f1e-9d55-b4b6ae1bc43e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:53:04.387797 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98cb5e95-1411-4f1e-9d55-b4b6ae1bc43e None admin] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:53:04.388052 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98cb5e95-1411-4f1e-9d55-b4b6ae1bc43e None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:53:04.388322 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98cb5e95-1411-4f1e-9d55-b4b6ae1bc43e None admin] PATH_INFO: `/v3/services` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:53:04.388965 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98cb5e95-1411-4f1e-9d55-b4b6ae1bc43e None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:53:04.390346 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98cb5e95-1411-4f1e-9d55-b4b6ae1bc43e None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:53:04.404722 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:53:04.404991 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 503/1014] 10.222.0.40 () {60 vars in 1141 bytes} [Fri Aug 26 10:53:04 2022] POST /identity/v3/services => generated 242 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 26 10:53:04.406807 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbc0362-6a0d-483a-b500-dab69bdccf31 None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:53:04.407092 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbc0362-6a0d-483a-b500-dab69bdccf31 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:53:04.407368 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbc0362-6a0d-483a-b500-dab69bdccf31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:53:04.427079 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-abbc0362-6a0d-483a-b500-dab69bdccf31 None None] MFA Rules not processed for user `ca155a48e2024c32af3fd2e19e5fac11`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 10:53:04.428232 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-abbc0362-6a0d-483a-b500-dab69bdccf31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 10:53:04.455330 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8216e908-0846-421e-9621-8ac7db9bd79f None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:53:04.494073 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f396565-cd90-4923-9a38-aef7946f4251 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:53:04.494709 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f396565-cd90-4923-9a38-aef7946f4251 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:53:04.495273 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f396565-cd90-4923-9a38-aef7946f4251 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:53:04.495572 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f396565-cd90-4923-9a38-aef7946f4251 None admin] PATH_INFO: `/v3/services` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:53:04.496158 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f396565-cd90-4923-9a38-aef7946f4251 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 10:53:04.496460 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f396565-cd90-4923-9a38-aef7946f4251 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:53:04.498193 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f396565-cd90-4923-9a38-aef7946f4251 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:53:04.505247 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 512/1015] 10.222.0.40 () {56 vars in 1112 bytes} [Fri Aug 26 10:53:04 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 10:53:04.511161 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f396565-cd90-4923-9a38-aef7946f4251 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:53:04.518285 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 504/1016] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 26 10:53:04 2022] POST /identity/v3/auth/tokens => generated 1813 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 26 10:53:04.616819 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f396565-cd90-4923-9a38-aef7946f4251 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:53:04.645265 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:53:04.651863 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aed80a3-e8b3-4506-a79e-d9cf6dc5b1b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:53:04.652466 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aed80a3-e8b3-4506-a79e-d9cf6dc5b1b9 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:53:04.652739 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aed80a3-e8b3-4506-a79e-d9cf6dc5b1b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:53:04.653005 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aed80a3-e8b3-4506-a79e-d9cf6dc5b1b9 None admin] PATH_INFO: `/v3/services` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:53:04.653629 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aed80a3-e8b3-4506-a79e-d9cf6dc5b1b9 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 10:53:04.653925 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aed80a3-e8b3-4506-a79e-d9cf6dc5b1b9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:53:04.655298 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aed80a3-e8b3-4506-a79e-d9cf6dc5b1b9 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:53:04.665460 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 513/1017] 10.222.0.40 () {56 vars in 1112 bytes} [Fri Aug 26 10:53:04 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 10:53:04.667035 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abbc0362-6a0d-483a-b500-dab69bdccf31 None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:53:04.715603 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:53:04.772573 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abbc0362-6a0d-483a-b500-dab69bdccf31 None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:53:04.795177 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aed80a3-e8b3-4506-a79e-d9cf6dc5b1b9 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:53:04.808210 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7ed5a0-3d8b-431c-88a3-be041426f8e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:53:04.809144 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7ed5a0-3d8b-431c-88a3-be041426f8e2 None admin] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:53:04.809433 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7ed5a0-3d8b-431c-88a3-be041426f8e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:53:04.809710 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7ed5a0-3d8b-431c-88a3-be041426f8e2 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:53:04.810575 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a7ed5a0-3d8b-431c-88a3-be041426f8e2 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:53:04.811947 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a7ed5a0-3d8b-431c-88a3-be041426f8e2 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:53:04.834691 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 505/1018] 10.222.0.40 () {60 vars in 1144 bytes} [Fri Aug 26 10:53:04 2022] POST /identity/v3/endpoints => generated 334 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 10:53:04.917427 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aed80a3-e8b3-4506-a79e-d9cf6dc5b1b9 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:53:04.964876 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599e513f-49e1-40be-9f02-a8d19e73fd2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:53:04.965522 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599e513f-49e1-40be-9f02-a8d19e73fd2a None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:53:04.965856 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599e513f-49e1-40be-9f02-a8d19e73fd2a None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:53:04.966102 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599e513f-49e1-40be-9f02-a8d19e73fd2a None admin] PATH_INFO: `/v3/domains/default` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:53:04.967248 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-599e513f-49e1-40be-9f02-a8d19e73fd2a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:53:04.968707 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-599e513f-49e1-40be-9f02-a8d19e73fd2a None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:53:04.971347 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 514/1019] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Aug 26 10:53:04 2022] GET /identity/v3/domains/default => generated 199 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 10:53:04.976960 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7ed5a0-3d8b-431c-88a3-be041426f8e2 None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:53:05.087327 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7ed5a0-3d8b-431c-88a3-be041426f8e2 None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:53:05.088937 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:53:05.126349 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7719ef4-299a-47e2-b54c-0e240bb52485 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:53:05.126963 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7719ef4-299a-47e2-b54c-0e240bb52485 None admin] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:53:05.127242 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7719ef4-299a-47e2-b54c-0e240bb52485 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:53:05.127514 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7719ef4-299a-47e2-b54c-0e240bb52485 None admin] PATH_INFO: `/v3/projects` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:53:05.128167 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7719ef4-299a-47e2-b54c-0e240bb52485 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:53:05.129595 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7719ef4-299a-47e2-b54c-0e240bb52485 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:53:05.159842 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 506/1020] 10.222.0.40 () {60 vars in 1141 bytes} [Fri Aug 26 10:53:04 2022] POST /identity/v3/projects => generated 311 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 10:53:06.360851 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51cda3a9-9f88-4f9e-9dca-23723d228ab9 None None] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:53:06.361365 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51cda3a9-9f88-4f9e-9dca-23723d228ab9 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:53:06.361365 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51cda3a9-9f88-4f9e-9dca-23723d228ab9 None None] PATH_INFO: `/` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:53:06.362820 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 515/1021] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 26 10:53:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 10:53:06.367941 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec222109-2b1c-4c42-87b3-7ffcd632e487 None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:53:06.368386 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec222109-2b1c-4c42-87b3-7ffcd632e487 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:53:06.368480 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec222109-2b1c-4c42-87b3-7ffcd632e487 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:53:06.387022 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-ec222109-2b1c-4c42-87b3-7ffcd632e487 None None] MFA Rules not processed for user `ca155a48e2024c32af3fd2e19e5fac11`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 10:53:06.388201 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-ec222109-2b1c-4c42-87b3-7ffcd632e487 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 10:53:06.455898 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 507/1022] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 26 10:53:06 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 26 10:53:06.731729 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51cda3a9-9f88-4f9e-9dca-23723d228ab9 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:53:06.794096 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:53:06.837205 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51cda3a9-9f88-4f9e-9dca-23723d228ab9 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:53:06.881145 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8a7854-827d-4a3a-b897-d57a72691539 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:53:06.881779 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8a7854-827d-4a3a-b897-d57a72691539 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:53:06.882083 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8a7854-827d-4a3a-b897-d57a72691539 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:53:06.882353 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8a7854-827d-4a3a-b897-d57a72691539 None admin] PATH_INFO: `/v3/services/image` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:53:06.882935 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd8a7854-827d-4a3a-b897-d57a72691539 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:53:06.884317 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd8a7854-827d-4a3a-b897-d57a72691539 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:53:06.893089 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application [None req-cd8a7854-827d-4a3a-b897-d57a72691539 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 26 10:53:06.893089 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 10:53:06.893089 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 26 10:53:06.893089 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 26 10:53:06.893089 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 26 10:53:06.893089 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 26 10:53:06.893089 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 26 10:53:06.893089 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 26 10:53:06.893089 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 26 10:53:06.893089 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Aug 26 10:53:06.893089 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 26 10:53:06.893089 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Aug 26 10:53:06.893089 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 10:53:06.893089 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 10:53:06.893089 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 10:53:06.893089 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 10:53:06.893089 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 10:53:06.894365 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 10:53:06.894365 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 10:53:06.894365 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 10:53:06.894365 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 10:53:06.894365 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 10:53:06.894365 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 10:53:06.894365 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 26 10:53:06.894365 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 26 10:53:06.894365 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 26 10:53:06.894365 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 26 10:53:06.894365 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 10:53:06.894365 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 10:53:06.894365 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 10:53:06.894365 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 10:53:06.894365 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 10:53:06.894365 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 10:53:06.895480 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 10:53:06.895480 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application with Lock( Aug 26 10:53:06.895480 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 10:53:06.895480 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._enter() Aug 26 10:53:06.895480 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 10:53:06.895480 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 10:53:06.895480 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 10:53:06.895480 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.creator() Aug 26 10:53:06.895480 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 10:53:06.895480 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application created_value = creator( Aug 26 10:53:06.895480 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 26 10:53:06.895480 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 26 10:53:06.895480 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 26 10:53:06.895480 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Aug 26 10:53:06.896551 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 516/1023] 10.222.0.40 () {56 vars in 1105 bytes} [Fri Aug 26 10:53:06 2022] GET /identity/v3/services/image => generated 86 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 10:53:06.900047 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec222109-2b1c-4c42-87b3-7ffcd632e487 None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:53:06.908775 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49c1395-35d3-4cc9-a65e-bd3d50823442 None None] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:53:06.909171 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49c1395-35d3-4cc9-a65e-bd3d50823442 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:53:06.909451 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49c1395-35d3-4cc9-a65e-bd3d50823442 None None] PATH_INFO: `/` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:53:06.910975 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 517/1024] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 26 10:53:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 26 10:53:06.915855 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12b4b59-4c3a-41f3-b55f-1139ea17c750 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:53:06.916138 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12b4b59-4c3a-41f3-b55f-1139ea17c750 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:53:06.916413 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12b4b59-4c3a-41f3-b55f-1139ea17c750 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:53:06.934925 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-e12b4b59-4c3a-41f3-b55f-1139ea17c750 None None] MFA Rules not processed for user `ca155a48e2024c32af3fd2e19e5fac11`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 10:53:06.936130 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-e12b4b59-4c3a-41f3-b55f-1139ea17c750 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 10:53:06.991785 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:53:07.005664 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 518/1025] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 26 10:53:06 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 26 10:53:07.010994 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec222109-2b1c-4c42-87b3-7ffcd632e487 None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:53:07.047005 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9931861c-5092-4f3a-b0cf-ef1ae18ad636 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:53:07.048113 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9931861c-5092-4f3a-b0cf-ef1ae18ad636 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:53:07.048391 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9931861c-5092-4f3a-b0cf-ef1ae18ad636 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:53:07.048667 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9931861c-5092-4f3a-b0cf-ef1ae18ad636 None admin] PATH_INFO: `/v3/services` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:53:07.049283 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9931861c-5092-4f3a-b0cf-ef1ae18ad636 None admin] RBAC: Adding query filter params (name=image) {{(pid=81423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 10:53:07.049582 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9931861c-5092-4f3a-b0cf-ef1ae18ad636 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:53:07.050984 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9931861c-5092-4f3a-b0cf-ef1ae18ad636 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:53:07.060542 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 508/1026] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Aug 26 10:53:06 2022] GET /identity/v3/services?name=image => generated 122 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 10:53:07.062109 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12b4b59-4c3a-41f3-b55f-1139ea17c750 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:53:07.100153 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b13a456-578d-43e2-9e5c-b0b687340d6a None None] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:53:07.100759 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b13a456-578d-43e2-9e5c-b0b687340d6a None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:53:07.101236 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b13a456-578d-43e2-9e5c-b0b687340d6a None None] PATH_INFO: `/` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:53:07.103087 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 509/1027] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 26 10:53:07 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 26 10:53:07.110337 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f777f4-da29-45f7-ba27-3deaa7dc6b02 None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:53:07.110626 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f777f4-da29-45f7-ba27-3deaa7dc6b02 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:53:07.110894 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f777f4-da29-45f7-ba27-3deaa7dc6b02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:53:07.121726 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:53:07.128853 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-b5f777f4-da29-45f7-ba27-3deaa7dc6b02 None None] MFA Rules not processed for user `ca155a48e2024c32af3fd2e19e5fac11`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 10:53:07.130074 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-b5f777f4-da29-45f7-ba27-3deaa7dc6b02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 10:53:07.170302 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12b4b59-4c3a-41f3-b55f-1139ea17c750 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:53:07.202024 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 510/1028] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 26 10:53:07 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 26 10:53:07.207509 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035f813c-ce71-434e-808d-7a1a6cfe2a3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:53:07.208322 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035f813c-ce71-434e-808d-7a1a6cfe2a3d None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:53:07.208778 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035f813c-ce71-434e-808d-7a1a6cfe2a3d None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:53:07.209226 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035f813c-ce71-434e-808d-7a1a6cfe2a3d None admin] PATH_INFO: `/v3/services` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:53:07.210105 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-035f813c-ce71-434e-808d-7a1a6cfe2a3d None admin] RBAC: Adding query filter params (type=image) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 10:53:07.210586 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-035f813c-ce71-434e-808d-7a1a6cfe2a3d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:53:07.212137 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-035f813c-ce71-434e-808d-7a1a6cfe2a3d None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:53:07.217519 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 519/1029] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Aug 26 10:53:07 2022] GET /identity/v3/services?type=image => generated 350 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 10:53:07.224156 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f777f4-da29-45f7-ba27-3deaa7dc6b02 None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:53:07.277422 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035f813c-ce71-434e-808d-7a1a6cfe2a3d None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:53:07.308714 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:53:07.332213 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f777f4-da29-45f7-ba27-3deaa7dc6b02 None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:53:07.370383 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe545989-6a58-4bf5-aaa8-5b9758e4df0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:53:07.371035 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe545989-6a58-4bf5-aaa8-5b9758e4df0c None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:53:07.371270 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe545989-6a58-4bf5-aaa8-5b9758e4df0c None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:53:07.371534 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe545989-6a58-4bf5-aaa8-5b9758e4df0c None admin] PATH_INFO: `/v3/endpoints` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:53:07.372591 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe545989-6a58-4bf5-aaa8-5b9758e4df0c None admin] RBAC: Adding query filter params (interface=public, service_id=84263d7c69104bca85f7e2d7075ad732, region_id=RegionOne) {{(pid=81423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 10:53:07.372897 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe545989-6a58-4bf5-aaa8-5b9758e4df0c None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:53:07.374607 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe545989-6a58-4bf5-aaa8-5b9758e4df0c None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:53:07.384062 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 511/1030] 10.222.0.40 () {56 vars in 1251 bytes} [Fri Aug 26 10:53:07 2022] GET /identity/v3/endpoints?service_id=84263d7c69104bca85f7e2d7075ad732&interface=public®ion_id=RegionOne => generated 194 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 10:53:07.385269 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe545989-6a58-4bf5-aaa8-5b9758e4df0c None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:53:07.398282 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035f813c-ce71-434e-808d-7a1a6cfe2a3d None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:53:07.417608 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:53:07.435571 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05df317a-8361-47a3-a736-8d32ac208aae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:53:07.436205 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05df317a-8361-47a3-a736-8d32ac208aae None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:53:07.436485 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05df317a-8361-47a3-a736-8d32ac208aae None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:53:07.436756 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05df317a-8361-47a3-a736-8d32ac208aae None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:53:07.449343 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05df317a-8361-47a3-a736-8d32ac208aae None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 10:53:07.455013 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application [None req-05df317a-8361-47a3-a736-8d32ac208aae None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 26 10:53:07.455013 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 10:53:07.455013 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 10:53:07.455013 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 10:53:07.455013 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 10:53:07.455013 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 10:53:07.455013 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 10:53:07.455013 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 10:53:07.455013 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 10:53:07.455013 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 10:53:07.455013 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 10:53:07.455013 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 10:53:07.455013 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 10:53:07.455013 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 10:53:07.455013 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 10:53:07.455013 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 10:53:07.455958 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 10:53:07.455958 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 10:53:07.455958 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 10:53:07.455958 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 10:53:07.455958 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 10:53:07.455958 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 10:53:07.455958 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 10:53:07.455958 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 10:53:07.455958 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 10:53:07.455958 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 10:53:07.455958 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 10:53:07.455958 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application with Lock( Aug 26 10:53:07.455958 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 10:53:07.455958 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._enter() Aug 26 10:53:07.455958 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 10:53:07.455958 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 10:53:07.456800 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 10:53:07.456800 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.creator() Aug 26 10:53:07.456800 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 10:53:07.456800 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application created_value = creator( Aug 26 10:53:07.456800 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 10:53:07.456800 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 10:53:07.456800 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 10:53:07.456800 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 10:53:07.456800 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 10:53:07.456800 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 10:53:07.456800 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 26 10:53:07.456800 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Aug 26 10:53:07.456800 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 520/1031] 10.222.0.40 () {56 vars in 1120 bytes} [Fri Aug 26 10:53:07 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 26 10:53:07.462721 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05df317a-8361-47a3-a736-8d32ac208aae None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:53:07.494653 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe545989-6a58-4bf5-aaa8-5b9758e4df0c None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:53:07.528732 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42bb5b13-b0fb-48cd-a81a-0a5e17f29719 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:53:07.529260 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42bb5b13-b0fb-48cd-a81a-0a5e17f29719 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:53:07.529539 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42bb5b13-b0fb-48cd-a81a-0a5e17f29719 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:53:07.529809 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42bb5b13-b0fb-48cd-a81a-0a5e17f29719 None admin] PATH_INFO: `/v3/services` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:53:07.530443 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42bb5b13-b0fb-48cd-a81a-0a5e17f29719 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:53:07.532499 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42bb5b13-b0fb-48cd-a81a-0a5e17f29719 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:53:07.538849 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 512/1032] 10.222.0.40 () {56 vars in 1087 bytes} [Fri Aug 26 10:53:07 2022] GET /identity/v3/services => generated 1957 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 10:53:07.539630 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42bb5b13-b0fb-48cd-a81a-0a5e17f29719 None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:53:07.560277 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:53:07.570951 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05df317a-8361-47a3-a736-8d32ac208aae None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:53:07.612100 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b416f10-a841-477d-bbdb-afc0356fded3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:53:07.612736 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b416f10-a841-477d-bbdb-afc0356fded3 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:53:07.613178 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b416f10-a841-477d-bbdb-afc0356fded3 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:53:07.613451 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b416f10-a841-477d-bbdb-afc0356fded3 None admin] PATH_INFO: `/v3/roles` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:53:07.614166 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b416f10-a841-477d-bbdb-afc0356fded3 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 10:53:07.614468 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b416f10-a841-477d-bbdb-afc0356fded3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:53:07.615864 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b416f10-a841-477d-bbdb-afc0356fded3 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:53:07.627723 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 521/1033] 10.222.0.40 () {56 vars in 1115 bytes} [Fri Aug 26 10:53:07 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 10:53:07.633931 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b416f10-a841-477d-bbdb-afc0356fded3 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:53:07.673044 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42bb5b13-b0fb-48cd-a81a-0a5e17f29719 None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:53:07.681689 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:53:07.718521 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2143daa5-613b-495b-84bd-639e44ca2395 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:53:07.719868 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2143daa5-613b-495b-84bd-639e44ca2395 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:53:07.720149 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2143daa5-613b-495b-84bd-639e44ca2395 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:53:07.720418 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2143daa5-613b-495b-84bd-639e44ca2395 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:53:07.721549 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2143daa5-613b-495b-84bd-639e44ca2395 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:53:07.723125 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2143daa5-613b-495b-84bd-639e44ca2395 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:53:07.726132 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 513/1034] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Aug 26 10:53:07 2022] GET /identity/v3/domains/default => generated 199 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 10:53:07.730635 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2143daa5-613b-495b-84bd-639e44ca2395 None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:53:07.749511 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b416f10-a841-477d-bbdb-afc0356fded3 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:53:07.788368 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e405310-1809-467e-93b1-7067f0bf6d17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:53:07.788908 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e405310-1809-467e-93b1-7067f0bf6d17 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:53:07.789177 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e405310-1809-467e-93b1-7067f0bf6d17 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:53:07.789442 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e405310-1809-467e-93b1-7067f0bf6d17 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:53:07.800830 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e405310-1809-467e-93b1-7067f0bf6d17 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:53:07.802373 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e405310-1809-467e-93b1-7067f0bf6d17 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:53:07.816869 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:53:07.817862 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application [None req-5e405310-1809-467e-93b1-7067f0bf6d17 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 10:53:07.817862 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 10:53:07.817862 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 10:53:07.817862 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 10:53:07.817862 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 10:53:07.817862 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 10:53:07.817862 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 10:53:07.817862 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 10:53:07.817862 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 10:53:07.817862 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 10:53:07.817862 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 10:53:07.817862 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 10:53:07.817862 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 10:53:07.817862 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 10:53:07.817862 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 10:53:07.817862 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 10:53:07.818817 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 10:53:07.818817 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 10:53:07.818817 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 10:53:07.818817 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 10:53:07.818817 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 10:53:07.818817 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 10:53:07.818817 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 10:53:07.818817 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application with Lock( Aug 26 10:53:07.818817 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 10:53:07.818817 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._enter() Aug 26 10:53:07.818817 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 10:53:07.818817 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 10:53:07.818817 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 10:53:07.818817 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.creator() Aug 26 10:53:07.818817 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 10:53:07.818817 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application created_value = creator( Aug 26 10:53:07.818817 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 10:53:07.819714 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 10:53:07.819714 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 10:53:07.819714 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Aug 26 10:53:07.819714 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 522/1035] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Aug 26 10:53:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 10:53:07.824766 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e405310-1809-467e-93b1-7067f0bf6d17 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:53:07.848577 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2143daa5-613b-495b-84bd-639e44ca2395 None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:53:07.890457 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c52b7a-bc2e-4308-8bd9-eddd6255e0e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:53:07.892319 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c52b7a-bc2e-4308-8bd9-eddd6255e0e8 None admin] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:53:07.892592 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c52b7a-bc2e-4308-8bd9-eddd6255e0e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:53:07.892860 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c52b7a-bc2e-4308-8bd9-eddd6255e0e8 None admin] PATH_INFO: `/v3/users` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:53:07.893497 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c52b7a-bc2e-4308-8bd9-eddd6255e0e8 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:53:07.894969 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c52b7a-bc2e-4308-8bd9-eddd6255e0e8 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:53:07.926228 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 514/1036] 10.222.0.40 () {60 vars in 1132 bytes} [Fri Aug 26 10:53:07 2022] POST /identity/v3/users => generated 285 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 26 10:53:07.927393 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:53:07.938720 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e405310-1809-467e-93b1-7067f0bf6d17 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:53:07.976786 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e6ee4e-d24c-4c40-b232-4c23e7a057ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:53:07.977414 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e6ee4e-d24c-4c40-b232-4c23e7a057ba None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:53:07.977693 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e6ee4e-d24c-4c40-b232-4c23e7a057ba None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:53:07.977966 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e6ee4e-d24c-4c40-b232-4c23e7a057ba None admin] PATH_INFO: `/v3/domains` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:53:07.978652 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e6ee4e-d24c-4c40-b232-4c23e7a057ba None admin] RBAC: Adding query filter params (name=Default) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 10:53:07.978946 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e6ee4e-d24c-4c40-b232-4c23e7a057ba None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:53:07.980356 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e6ee4e-d24c-4c40-b232-4c23e7a057ba None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:53:07.999199 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 523/1037] 10.222.0.40 () {56 vars in 1109 bytes} [Fri Aug 26 10:53:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 10:53:08.001421 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c52b7a-bc2e-4308-8bd9-eddd6255e0e8 None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:53:08.115869 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c52b7a-bc2e-4308-8bd9-eddd6255e0e8 None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:53:08.146765 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:53:08.154505 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e18fd9e-8de7-470e-be16-a98ff9704c97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:53:08.155140 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e18fd9e-8de7-470e-be16-a98ff9704c97 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:53:08.155419 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e18fd9e-8de7-470e-be16-a98ff9704c97 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:53:08.155683 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e18fd9e-8de7-470e-be16-a98ff9704c97 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:53:08.165013 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e18fd9e-8de7-470e-be16-a98ff9704c97 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:53:08.166672 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e18fd9e-8de7-470e-be16-a98ff9704c97 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:53:08.179563 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application [None req-6e18fd9e-8de7-470e-be16-a98ff9704c97 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 26 10:53:08.179563 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 10:53:08.179563 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 10:53:08.179563 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 10:53:08.179563 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 10:53:08.179563 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 10:53:08.179563 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 10:53:08.179563 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 10:53:08.179563 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 10:53:08.179563 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 10:53:08.179563 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 10:53:08.179563 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 10:53:08.179563 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 26 10:53:08.179563 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 26 10:53:08.179563 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 26 10:53:08.179563 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 26 10:53:08.180636 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 10:53:08.180636 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 10:53:08.180636 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 26 10:53:08.180636 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 10:53:08.180636 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 26 10:53:08.180636 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 10:53:08.180636 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 10:53:08.180636 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 10:53:08.180636 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 10:53:08.180636 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 10:53:08.180636 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 10:53:08.180636 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application with Lock( Aug 26 10:53:08.180636 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 10:53:08.180636 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._enter() Aug 26 10:53:08.180636 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 10:53:08.180636 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 10:53:08.180636 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 10:53:08.180636 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.creator() Aug 26 10:53:08.181609 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 10:53:08.181609 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application created_value = creator( Aug 26 10:53:08.181609 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 26 10:53:08.181609 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 26 10:53:08.181609 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 26 10:53:08.181609 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 26 10:53:08.181609 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 26 10:53:08.181609 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 26 10:53:08.181609 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 26 10:53:08.181609 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Aug 26 10:53:08.181609 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 515/1038] 10.222.0.40 () {56 vars in 1093 bytes} [Fri Aug 26 10:53:07 2022] GET /identity/v3/users/nova => generated 82 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 10:53:08.186737 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e6ee4e-d24c-4c40-b232-4c23e7a057ba None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:53:08.290519 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e6ee4e-d24c-4c40-b232-4c23e7a057ba None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:53:08.324601 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75fae18c-f7b2-4744-9bed-a3c580c578ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:53:08.325201 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75fae18c-f7b2-4744-9bed-a3c580c578ab None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:53:08.325473 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75fae18c-f7b2-4744-9bed-a3c580c578ab None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:53:08.325739 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75fae18c-f7b2-4744-9bed-a3c580c578ab None admin] PATH_INFO: `/v3/users` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:53:08.326483 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75fae18c-f7b2-4744-9bed-a3c580c578ab None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 10:53:08.326783 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75fae18c-f7b2-4744-9bed-a3c580c578ab None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:53:08.328803 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75fae18c-f7b2-4744-9bed-a3c580c578ab None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:53:08.351158 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 524/1039] 10.222.0.40 () {56 vars in 1133 bytes} [Fri Aug 26 10:53:08 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 10:53:08.352671 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e18fd9e-8de7-470e-be16-a98ff9704c97 None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:53:08.459664 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e18fd9e-8de7-470e-be16-a98ff9704c97 None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:53:08.493651 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1994838-412a-4916-b91d-34be99c7d414 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:53:08.494305 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1994838-412a-4916-b91d-34be99c7d414 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:53:08.494587 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1994838-412a-4916-b91d-34be99c7d414 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:53:08.494860 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1994838-412a-4916-b91d-34be99c7d414 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:53:08.554871 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:53:08.567809 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1994838-412a-4916-b91d-34be99c7d414 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:53:08.569262 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1994838-412a-4916-b91d-34be99c7d414 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:53:08.676880 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 516/1040] 10.222.0.40 () {58 vars in 1282 bytes} [Fri Aug 26 10:53:08 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 26 10:53:08.678220 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75fae18c-f7b2-4744-9bed-a3c580c578ab None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:53:08.691944 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:53:08.781129 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75fae18c-f7b2-4744-9bed-a3c580c578ab None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:53:08.815315 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02922710-bbb4-416f-b2ee-cf160ba6638e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:53:08.815940 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02922710-bbb4-416f-b2ee-cf160ba6638e None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:53:08.816218 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02922710-bbb4-416f-b2ee-cf160ba6638e None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:53:08.816492 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02922710-bbb4-416f-b2ee-cf160ba6638e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:53:08.826290 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02922710-bbb4-416f-b2ee-cf160ba6638e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:53:08.827741 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02922710-bbb4-416f-b2ee-cf160ba6638e None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:53:08.842183 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application [None req-02922710-bbb4-416f-b2ee-cf160ba6638e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 10:53:08.842183 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 10:53:08.842183 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 10:53:08.842183 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 10:53:08.842183 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 10:53:08.842183 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 10:53:08.842183 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 10:53:08.842183 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 10:53:08.842183 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 10:53:08.842183 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 10:53:08.842183 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 10:53:08.842183 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 10:53:08.842183 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 10:53:08.842183 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 10:53:08.842183 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 10:53:08.842183 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 10:53:08.843189 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 10:53:08.843189 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 10:53:08.843189 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 10:53:08.843189 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 10:53:08.843189 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 10:53:08.843189 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 10:53:08.843189 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 10:53:08.843189 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application with Lock( Aug 26 10:53:08.843189 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 10:53:08.843189 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._enter() Aug 26 10:53:08.843189 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 10:53:08.843189 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 10:53:08.843189 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 10:53:08.843189 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.creator() Aug 26 10:53:08.843189 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 10:53:08.843189 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application created_value = creator( Aug 26 10:53:08.843189 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 10:53:08.844264 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 10:53:08.844264 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 10:53:08.844264 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Aug 26 10:53:08.846311 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 525/1041] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Aug 26 10:53:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 10:53:08.848937 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1994838-412a-4916-b91d-34be99c7d414 None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:53:08.946447 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:53:08.957553 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1994838-412a-4916-b91d-34be99c7d414 None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:53:08.993425 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df0110c-a52d-49c9-af6e-e2ac941b7bbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:53:08.994061 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df0110c-a52d-49c9-af6e-e2ac941b7bbe None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:53:08.994341 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df0110c-a52d-49c9-af6e-e2ac941b7bbe None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:53:08.994611 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df0110c-a52d-49c9-af6e-e2ac941b7bbe None admin] PATH_INFO: `/v3/domains` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:53:08.995253 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df0110c-a52d-49c9-af6e-e2ac941b7bbe None admin] RBAC: Adding query filter params (name=Default) {{(pid=81423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 10:53:08.995553 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df0110c-a52d-49c9-af6e-e2ac941b7bbe None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:53:08.996935 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df0110c-a52d-49c9-af6e-e2ac941b7bbe None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:53:09.013775 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 517/1042] 10.222.0.40 () {56 vars in 1109 bytes} [Fri Aug 26 10:53:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 10:53:09.015283 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02922710-bbb4-416f-b2ee-cf160ba6638e None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:53:09.080874 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:53:09.123203 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02922710-bbb4-416f-b2ee-cf160ba6638e None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:53:09.156137 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c485423b-ac7c-4651-af18-014f4d266457 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:53:09.157289 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c485423b-ac7c-4651-af18-014f4d266457 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:53:09.157564 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c485423b-ac7c-4651-af18-014f4d266457 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:53:09.157831 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c485423b-ac7c-4651-af18-014f4d266457 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:53:09.164444 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c485423b-ac7c-4651-af18-014f4d266457 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:53:09.165884 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c485423b-ac7c-4651-af18-014f4d266457 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:53:09.177631 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application [None req-c485423b-ac7c-4651-af18-014f4d266457 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 10:53:09.177631 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 10:53:09.177631 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 10:53:09.177631 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 10:53:09.177631 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 10:53:09.177631 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 10:53:09.177631 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 10:53:09.177631 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 10:53:09.177631 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 10:53:09.177631 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 10:53:09.177631 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 10:53:09.177631 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 10:53:09.177631 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 26 10:53:09.177631 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 26 10:53:09.177631 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 26 10:53:09.178763 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 26 10:53:09.178763 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 10:53:09.178763 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 10:53:09.178763 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 10:53:09.178763 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 10:53:09.178763 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 10:53:09.178763 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 10:53:09.178763 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 10:53:09.178763 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application with Lock( Aug 26 10:53:09.178763 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 10:53:09.178763 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._enter() Aug 26 10:53:09.178763 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 10:53:09.178763 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 10:53:09.178763 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 10:53:09.178763 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.creator() Aug 26 10:53:09.178763 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 10:53:09.178763 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application created_value = creator( Aug 26 10:53:09.180132 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 26 10:53:09.180132 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 26 10:53:09.180132 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 26 10:53:09.180132 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 26 10:53:09.180132 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 26 10:53:09.180132 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 26 10:53:09.180132 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 10:53:09.180132 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Aug 26 10:53:09.180851 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 526/1043] 10.222.0.40 () {56 vars in 1111 bytes} [Fri Aug 26 10:53:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 26 10:53:09.184620 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df0110c-a52d-49c9-af6e-e2ac941b7bbe None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:53:09.287288 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df0110c-a52d-49c9-af6e-e2ac941b7bbe None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:53:09.320821 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a67d1c0-8fc4-433c-87f8-539e7e0f8bb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:53:09.321438 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a67d1c0-8fc4-433c-87f8-539e7e0f8bb9 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:53:09.321714 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a67d1c0-8fc4-433c-87f8-539e7e0f8bb9 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:53:09.322002 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a67d1c0-8fc4-433c-87f8-539e7e0f8bb9 None admin] PATH_INFO: `/v3/projects` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:53:09.322696 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a67d1c0-8fc4-433c-87f8-539e7e0f8bb9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 10:53:09.322996 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a67d1c0-8fc4-433c-87f8-539e7e0f8bb9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:53:09.324421 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a67d1c0-8fc4-433c-87f8-539e7e0f8bb9 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:53:09.341538 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 518/1044] 10.222.0.40 () {56 vars in 1148 bytes} [Fri Aug 26 10:53:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 10:53:09.342750 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c485423b-ac7c-4651-af18-014f4d266457 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:53:09.446904 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c485423b-ac7c-4651-af18-014f4d266457 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:53:09.471377 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:53:09.483339 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3879a8f-d9c4-4680-9e96-79dc47daf729 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:53:09.484203 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3879a8f-d9c4-4680-9e96-79dc47daf729 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:53:09.484510 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3879a8f-d9c4-4680-9e96-79dc47daf729 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:53:09.484782 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3879a8f-d9c4-4680-9e96-79dc47daf729 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:53:09.485502 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3879a8f-d9c4-4680-9e96-79dc47daf729 None admin] RBAC: Adding query filter params (role.id=3f7d20781437466aabcbf11bda248f3d, scope.project.id=0e30ccf0c1204727b786876c141dbcd2, user.id=1f4b4a662db049ecacc865ecff10bdb0) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 10:53:09.485805 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3879a8f-d9c4-4680-9e96-79dc47daf729 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:53:09.487248 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3879a8f-d9c4-4680-9e96-79dc47daf729 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:53:09.502080 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 527/1045] 10.222.0.40 () {56 vars in 1374 bytes} [Fri Aug 26 10:53:09 2022] GET /identity/v3/role_assignments?user.id=1f4b4a662db049ecacc865ecff10bdb0&scope.project.id=0e30ccf0c1204727b786876c141dbcd2&role.id=3f7d20781437466aabcbf11bda248f3d => generated 259 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 10:53:09.502284 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde464c0-4cce-4dcd-ab2d-57c492c0b223 None None] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:53:09.502284 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde464c0-4cce-4dcd-ab2d-57c492c0b223 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:53:09.502284 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde464c0-4cce-4dcd-ab2d-57c492c0b223 None None] PATH_INFO: `/` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:53:09.502284 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 519/1046] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 26 10:53:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 10:53:09.506285 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c238948e-da9e-4e4d-b3e0-14034dcca603 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:53:09.506571 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c238948e-da9e-4e4d-b3e0-14034dcca603 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:53:09.506848 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c238948e-da9e-4e4d-b3e0-14034dcca603 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:53:09.526769 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-c238948e-da9e-4e4d-b3e0-14034dcca603 None None] MFA Rules not processed for user `ca155a48e2024c32af3fd2e19e5fac11`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 10:53:09.527999 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-c238948e-da9e-4e4d-b3e0-14034dcca603 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 10:53:09.604292 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 528/1047] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 26 10:53:09 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 26 10:53:09.881608 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde464c0-4cce-4dcd-ab2d-57c492c0b223 None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:53:09.895532 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:53:09.930020 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e704e0a3-31be-4f4b-bdba-df81795b4c31 None None] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:53:09.930535 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e704e0a3-31be-4f4b-bdba-df81795b4c31 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:53:09.930626 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e704e0a3-31be-4f4b-bdba-df81795b4c31 None None] PATH_INFO: `/` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:53:09.934314 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 529/1048] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 26 10:53:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 26 10:53:09.936770 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455f5d3d-6de5-4f4a-9952-40db267e08eb None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:53:09.937269 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455f5d3d-6de5-4f4a-9952-40db267e08eb None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:53:09.938352 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455f5d3d-6de5-4f4a-9952-40db267e08eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:53:09.958931 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-455f5d3d-6de5-4f4a-9952-40db267e08eb None None] MFA Rules not processed for user `ca155a48e2024c32af3fd2e19e5fac11`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 10:53:09.960575 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-455f5d3d-6de5-4f4a-9952-40db267e08eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 10:53:09.995828 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde464c0-4cce-4dcd-ab2d-57c492c0b223 None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:53:10.031306 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 530/1049] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 26 10:53:09 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 26 10:53:10.031609 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b643e6-135a-478d-badf-a1dadebc5616 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:53:10.032136 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b643e6-135a-478d-badf-a1dadebc5616 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:53:10.032410 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b643e6-135a-478d-badf-a1dadebc5616 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:53:10.032667 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b643e6-135a-478d-badf-a1dadebc5616 None admin] PATH_INFO: `/v3/services/image` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:53:10.033248 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b643e6-135a-478d-badf-a1dadebc5616 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:53:10.034628 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b643e6-135a-478d-badf-a1dadebc5616 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:53:10.045481 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application [None req-73b643e6-135a-478d-badf-a1dadebc5616 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 26 10:53:10.045481 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 10:53:10.045481 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 26 10:53:10.045481 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 26 10:53:10.045481 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 26 10:53:10.045481 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 26 10:53:10.045481 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 26 10:53:10.045481 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 26 10:53:10.045481 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 26 10:53:10.045481 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Aug 26 10:53:10.045481 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 26 10:53:10.045481 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Aug 26 10:53:10.045481 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 10:53:10.045481 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 10:53:10.045481 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 10:53:10.045481 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 10:53:10.045481 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 10:53:10.046550 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 10:53:10.046550 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 10:53:10.046550 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 10:53:10.046550 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 10:53:10.046550 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 10:53:10.046550 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 10:53:10.046550 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 26 10:53:10.046550 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 26 10:53:10.046550 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 26 10:53:10.046550 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 26 10:53:10.046550 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 10:53:10.046550 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 10:53:10.046550 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 10:53:10.046550 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 10:53:10.046550 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 10:53:10.046550 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 10:53:10.047404 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 10:53:10.047404 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application with Lock( Aug 26 10:53:10.047404 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 10:53:10.047404 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._enter() Aug 26 10:53:10.047404 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 10:53:10.047404 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 10:53:10.047404 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 10:53:10.047404 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.creator() Aug 26 10:53:10.047404 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 10:53:10.047404 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application created_value = creator( Aug 26 10:53:10.047404 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 26 10:53:10.047404 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 26 10:53:10.047404 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 26 10:53:10.047404 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Aug 26 10:53:10.048974 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 520/1050] 10.222.0.40 () {56 vars in 1105 bytes} [Fri Aug 26 10:53:09 2022] GET /identity/v3/services/image => generated 86 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 10:53:10.052532 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455f5d3d-6de5-4f4a-9952-40db267e08eb None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:53:10.100278 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:53:10.154479 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455f5d3d-6de5-4f4a-9952-40db267e08eb None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:53:10.188607 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb8ed16-73c7-4622-9e50-8bb3941df92d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:53:10.189204 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb8ed16-73c7-4622-9e50-8bb3941df92d None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:53:10.189624 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb8ed16-73c7-4622-9e50-8bb3941df92d None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:53:10.189813 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb8ed16-73c7-4622-9e50-8bb3941df92d None admin] PATH_INFO: `/v3/services` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:53:10.190569 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb8ed16-73c7-4622-9e50-8bb3941df92d None admin] RBAC: Adding query filter params (name=image) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 10:53:10.190872 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb8ed16-73c7-4622-9e50-8bb3941df92d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:53:10.192232 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb8ed16-73c7-4622-9e50-8bb3941df92d None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:53:10.197096 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 531/1051] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Aug 26 10:53:10 2022] GET /identity/v3/services?name=image => generated 122 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 10:53:10.202867 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b643e6-135a-478d-badf-a1dadebc5616 None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:53:10.280649 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:53:10.307105 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b643e6-135a-478d-badf-a1dadebc5616 None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:53:10.307873 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb8ed16-73c7-4622-9e50-8bb3941df92d None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:53:10.345687 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eaa0c30-3e16-4807-abf3-180cc32b7f1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:53:10.346361 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eaa0c30-3e16-4807-abf3-180cc32b7f1a None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:53:10.346644 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eaa0c30-3e16-4807-abf3-180cc32b7f1a None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:53:10.346915 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eaa0c30-3e16-4807-abf3-180cc32b7f1a None admin] PATH_INFO: `/v3/services` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:53:10.347598 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eaa0c30-3e16-4807-abf3-180cc32b7f1a None admin] RBAC: Adding query filter params (type=image) {{(pid=81423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 10:53:10.347904 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eaa0c30-3e16-4807-abf3-180cc32b7f1a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:53:10.349705 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eaa0c30-3e16-4807-abf3-180cc32b7f1a None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:53:10.356403 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 521/1052] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Aug 26 10:53:10 2022] GET /identity/v3/services?type=image => generated 350 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 10:53:10.363594 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eaa0c30-3e16-4807-abf3-180cc32b7f1a None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:53:10.414772 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb8ed16-73c7-4622-9e50-8bb3941df92d None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:53:10.426928 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:53:10.454461 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5eedd9-6839-4f48-8d35-409326a86b81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:53:10.455090 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5eedd9-6839-4f48-8d35-409326a86b81 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:53:10.455365 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5eedd9-6839-4f48-8d35-409326a86b81 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:53:10.455635 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5eedd9-6839-4f48-8d35-409326a86b81 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:53:10.466706 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d5eedd9-6839-4f48-8d35-409326a86b81 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 10:53:10.471731 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eaa0c30-3e16-4807-abf3-180cc32b7f1a None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:53:10.472000 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application [None req-9d5eedd9-6839-4f48-8d35-409326a86b81 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 26 10:53:10.472000 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 10:53:10.472000 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 10:53:10.472000 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 10:53:10.472000 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 10:53:10.472000 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 10:53:10.472000 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 10:53:10.472000 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 10:53:10.472000 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 10:53:10.472000 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 10:53:10.472000 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 10:53:10.472000 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 10:53:10.472000 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 10:53:10.472000 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 10:53:10.472000 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 10:53:10.472000 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 10:53:10.472957 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 10:53:10.472957 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 10:53:10.472957 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 10:53:10.472957 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 10:53:10.472957 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 10:53:10.472957 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 10:53:10.472957 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 10:53:10.472957 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 10:53:10.472957 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 10:53:10.472957 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 10:53:10.472957 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 10:53:10.472957 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application with Lock( Aug 26 10:53:10.472957 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 10:53:10.472957 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._enter() Aug 26 10:53:10.472957 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 10:53:10.472957 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 10:53:10.475012 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 10:53:10.475012 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.creator() Aug 26 10:53:10.475012 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 10:53:10.475012 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application created_value = creator( Aug 26 10:53:10.475012 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 10:53:10.475012 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 10:53:10.475012 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 10:53:10.475012 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 10:53:10.475012 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 10:53:10.475012 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 10:53:10.475012 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 26 10:53:10.475012 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Aug 26 10:53:10.475012 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 532/1053] 10.222.0.40 () {56 vars in 1096 bytes} [Fri Aug 26 10:53:10 2022] GET /identity/v3/roles/admin => generated 83 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 26 10:53:10.478847 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5eedd9-6839-4f48-8d35-409326a86b81 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:53:10.515710 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4951703-a7ec-4694-9fe1-9d6bada30e6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:53:10.516512 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4951703-a7ec-4694-9fe1-9d6bada30e6b None admin] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:53:10.516796 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4951703-a7ec-4694-9fe1-9d6bada30e6b None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:53:10.517067 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4951703-a7ec-4694-9fe1-9d6bada30e6b None admin] PATH_INFO: `/v3/endpoints` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:53:10.517724 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4951703-a7ec-4694-9fe1-9d6bada30e6b None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:53:10.519118 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4951703-a7ec-4694-9fe1-9d6bada30e6b None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:53:10.527690 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:53:10.545327 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 522/1054] 10.222.0.40 () {60 vars in 1144 bytes} [Fri Aug 26 10:53:10 2022] POST /identity/v3/endpoints => generated 327 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 26 10:53:10.591510 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5eedd9-6839-4f48-8d35-409326a86b81 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:53:10.636665 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19984900-a2aa-4d78-a0ee-a6f2b1ccfccc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:53:10.637285 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19984900-a2aa-4d78-a0ee-a6f2b1ccfccc None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:53:10.637555 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19984900-a2aa-4d78-a0ee-a6f2b1ccfccc None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:53:10.637827 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19984900-a2aa-4d78-a0ee-a6f2b1ccfccc None admin] PATH_INFO: `/v3/roles` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:53:10.638529 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19984900-a2aa-4d78-a0ee-a6f2b1ccfccc None admin] RBAC: Adding query filter params (name=admin) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 10:53:10.638830 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19984900-a2aa-4d78-a0ee-a6f2b1ccfccc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:53:10.640279 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19984900-a2aa-4d78-a0ee-a6f2b1ccfccc None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:53:10.654069 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 533/1055] 10.222.0.40 () {56 vars in 1099 bytes} [Fri Aug 26 10:53:10 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 10:53:10.657117 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4951703-a7ec-4694-9fe1-9d6bada30e6b None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:53:10.767738 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4951703-a7ec-4694-9fe1-9d6bada30e6b None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:53:10.803453 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc4b712-bacf-473c-a567-f46dcc4bd7d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:53:10.804437 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc4b712-bacf-473c-a567-f46dcc4bd7d9 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:53:10.804896 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc4b712-bacf-473c-a567-f46dcc4bd7d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:53:10.805373 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc4b712-bacf-473c-a567-f46dcc4bd7d9 None admin] PATH_INFO: `/v3/users/76e608b7f48c4fe6a152d759ac0f648f` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:53:10.819608 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc4b712-bacf-473c-a567-f46dcc4bd7d9 None admin] RBAC: Authorizing `identity:get_user(user_id=76e608b7f48c4fe6a152d759ac0f648f)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:53:10.821390 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc4b712-bacf-473c-a567-f46dcc4bd7d9 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:53:10.823617 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 523/1056] 10.222.0.40 () {56 vars in 1177 bytes} [Fri Aug 26 10:53:10 2022] GET /identity/v3/users/76e608b7f48c4fe6a152d759ac0f648f => generated 285 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 10:53:10.829304 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19984900-a2aa-4d78-a0ee-a6f2b1ccfccc None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:53:10.865034 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:53:10.943014 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19984900-a2aa-4d78-a0ee-a6f2b1ccfccc None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:53:10.977991 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e4224d-7e10-4b32-8453-3ba7851005a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:53:10.978641 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e4224d-7e10-4b32-8453-3ba7851005a7 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:53:10.978923 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e4224d-7e10-4b32-8453-3ba7851005a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:53:10.979192 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e4224d-7e10-4b32-8453-3ba7851005a7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:53:11.049638 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e4224d-7e10-4b32-8453-3ba7851005a7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:53:11.051090 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e4224d-7e10-4b32-8453-3ba7851005a7 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:53:11.078001 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:53:11.154424 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 534/1057] 10.222.0.40 () {58 vars in 1282 bytes} [Fri Aug 26 10:53:10 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 26 10:53:11.160724 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc4b712-bacf-473c-a567-f46dcc4bd7d9 None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:53:11.216276 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:53:11.268355 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc4b712-bacf-473c-a567-f46dcc4bd7d9 None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:53:11.303192 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26886dcf-b717-47c5-919c-68f5bcb8956f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:53:11.303743 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26886dcf-b717-47c5-919c-68f5bcb8956f None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:53:11.304015 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26886dcf-b717-47c5-919c-68f5bcb8956f None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:53:11.304282 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26886dcf-b717-47c5-919c-68f5bcb8956f None admin] PATH_INFO: `/v3/projects/647d6df216314e948b277a186b28a0ac` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:53:11.305384 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26886dcf-b717-47c5-919c-68f5bcb8956f None admin] RBAC: Authorizing `identity:get_project(project_id=647d6df216314e948b277a186b28a0ac)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:53:11.306818 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26886dcf-b717-47c5-919c-68f5bcb8956f None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:53:11.314119 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 524/1058] 10.222.0.40 () {56 vars in 1186 bytes} [Fri Aug 26 10:53:11 2022] GET /identity/v3/projects/647d6df216314e948b277a186b28a0ac => generated 311 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 10:53:11.315630 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e4224d-7e10-4b32-8453-3ba7851005a7 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:53:11.382561 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6fd3e3-4e61-4285-90e7-dfd33a60d40b None None] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:53:11.382897 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6fd3e3-4e61-4285-90e7-dfd33a60d40b None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:53:11.383103 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6fd3e3-4e61-4285-90e7-dfd33a60d40b None None] PATH_INFO: `/` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:53:11.384572 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 525/1059] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 26 10:53:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 26 10:53:11.389068 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e956c6d-9b87-48df-957b-227c2fbda77f None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:53:11.389346 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e956c6d-9b87-48df-957b-227c2fbda77f None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:53:11.389622 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e956c6d-9b87-48df-957b-227c2fbda77f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:53:11.408135 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-8e956c6d-9b87-48df-957b-227c2fbda77f None None] MFA Rules not processed for user `ca155a48e2024c32af3fd2e19e5fac11`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 10:53:11.409334 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-8e956c6d-9b87-48df-957b-227c2fbda77f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 10:53:11.423546 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e4224d-7e10-4b32-8453-3ba7851005a7 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:53:11.455861 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:53:11.465159 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8d7ec6-ad23-4825-a9a7-3f8f34d75319 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:53:11.465757 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8d7ec6-ad23-4825-a9a7-3f8f34d75319 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:53:11.466062 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8d7ec6-ad23-4825-a9a7-3f8f34d75319 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:53:11.466330 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8d7ec6-ad23-4825-a9a7-3f8f34d75319 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:53:11.467041 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af8d7ec6-ad23-4825-a9a7-3f8f34d75319 None admin] RBAC: Adding query filter params (role.id=1f97e4602e2e4d6498913a81f9ae8d6a, scope.project.id=647d6df216314e948b277a186b28a0ac, user.id=76e608b7f48c4fe6a152d759ac0f648f) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 10:53:11.467341 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af8d7ec6-ad23-4825-a9a7-3f8f34d75319 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:53:11.468830 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af8d7ec6-ad23-4825-a9a7-3f8f34d75319 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:53:11.474567 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 535/1060] 10.222.0.40 () {56 vars in 1374 bytes} [Fri Aug 26 10:53:11 2022] GET /identity/v3/role_assignments?user.id=76e608b7f48c4fe6a152d759ac0f648f&scope.project.id=647d6df216314e948b277a186b28a0ac&role.id=1f97e4602e2e4d6498913a81f9ae8d6a => generated 259 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 10:53:11.481855 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 526/1061] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 26 10:53:11 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 26 10:53:11.754841 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8d7ec6-ad23-4825-a9a7-3f8f34d75319 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:53:11.859649 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8d7ec6-ad23-4825-a9a7-3f8f34d75319 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:53:11.895511 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36ad560-3536-48dd-92a6-9e9cd7e7db6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:53:11.897771 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36ad560-3536-48dd-92a6-9e9cd7e7db6a None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:53:11.897771 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36ad560-3536-48dd-92a6-9e9cd7e7db6a None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:53:11.897771 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36ad560-3536-48dd-92a6-9e9cd7e7db6a None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:53:11.908159 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c36ad560-3536-48dd-92a6-9e9cd7e7db6a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 10:53:11.913342 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application [None req-c36ad560-3536-48dd-92a6-9e9cd7e7db6a None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 26 10:53:11.913342 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 10:53:11.913342 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 10:53:11.913342 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 10:53:11.913342 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 10:53:11.913342 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 10:53:11.913342 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 10:53:11.913342 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 10:53:11.913342 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 10:53:11.913342 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 10:53:11.913342 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 10:53:11.913342 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 10:53:11.913342 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 10:53:11.913342 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 10:53:11.913342 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 10:53:11.913342 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 10:53:11.918085 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 10:53:11.918085 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 10:53:11.918085 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 10:53:11.918085 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 10:53:11.918085 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 10:53:11.918085 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 10:53:11.918085 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 10:53:11.918085 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 10:53:11.918085 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 10:53:11.918085 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 10:53:11.918085 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 10:53:11.918085 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application with Lock( Aug 26 10:53:11.918085 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 10:53:11.918085 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._enter() Aug 26 10:53:11.918085 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 10:53:11.918085 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 10:53:11.918998 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 10:53:11.918998 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.creator() Aug 26 10:53:11.918998 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 10:53:11.918998 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application created_value = creator( Aug 26 10:53:11.918998 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 10:53:11.918998 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 10:53:11.918998 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 10:53:11.918998 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 10:53:11.918998 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 10:53:11.918998 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 10:53:11.918998 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 26 10:53:11.918998 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Aug 26 10:53:11.918998 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 536/1062] 10.222.0.40 () {56 vars in 1120 bytes} [Fri Aug 26 10:53:11 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 26 10:53:11.921110 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e956c6d-9b87-48df-957b-227c2fbda77f None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:53:11.974033 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:53:12.026181 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e956c6d-9b87-48df-957b-227c2fbda77f None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:53:12.060567 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f92ac9-1086-4b6c-a4d1-1774bf9c4782 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:53:12.061169 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f92ac9-1086-4b6c-a4d1-1774bf9c4782 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:53:12.061448 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f92ac9-1086-4b6c-a4d1-1774bf9c4782 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:53:12.061716 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f92ac9-1086-4b6c-a4d1-1774bf9c4782 None admin] PATH_INFO: `/v3/roles` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:53:12.062408 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f92ac9-1086-4b6c-a4d1-1774bf9c4782 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 10:53:12.063253 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f92ac9-1086-4b6c-a4d1-1774bf9c4782 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:53:12.064609 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f92ac9-1086-4b6c-a4d1-1774bf9c4782 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:53:12.077671 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 527/1063] 10.222.0.40 () {56 vars in 1115 bytes} [Fri Aug 26 10:53:11 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 10:53:12.079547 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36ad560-3536-48dd-92a6-9e9cd7e7db6a None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:53:12.135833 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:53:12.184907 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36ad560-3536-48dd-92a6-9e9cd7e7db6a None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:53:12.218105 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2981945-3895-4c7a-b0e9-46c6e3f8307b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:53:12.219255 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2981945-3895-4c7a-b0e9-46c6e3f8307b None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:53:12.219700 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2981945-3895-4c7a-b0e9-46c6e3f8307b None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:53:12.220109 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2981945-3895-4c7a-b0e9-46c6e3f8307b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:53:12.231328 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2981945-3895-4c7a-b0e9-46c6e3f8307b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:53:12.232754 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2981945-3895-4c7a-b0e9-46c6e3f8307b None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:53:12.247731 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application [None req-a2981945-3895-4c7a-b0e9-46c6e3f8307b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 10:53:12.247731 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 10:53:12.247731 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 10:53:12.247731 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 10:53:12.247731 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 10:53:12.247731 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 10:53:12.247731 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 10:53:12.247731 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 10:53:12.247731 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 10:53:12.247731 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 10:53:12.247731 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 10:53:12.247731 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 10:53:12.247731 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 10:53:12.247731 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 10:53:12.247731 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 10:53:12.247731 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 10:53:12.248724 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 10:53:12.248724 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 10:53:12.248724 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 10:53:12.248724 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 10:53:12.248724 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 10:53:12.248724 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 10:53:12.248724 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 10:53:12.248724 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application with Lock( Aug 26 10:53:12.248724 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 10:53:12.248724 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._enter() Aug 26 10:53:12.248724 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 10:53:12.248724 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 10:53:12.248724 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 10:53:12.248724 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.creator() Aug 26 10:53:12.248724 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 10:53:12.248724 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application created_value = creator( Aug 26 10:53:12.248724 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 10:53:12.254386 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 10:53:12.254386 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 10:53:12.254386 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Aug 26 10:53:12.254386 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 537/1064] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Aug 26 10:53:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 10:53:12.256170 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f92ac9-1086-4b6c-a4d1-1774bf9c4782 None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:53:12.359092 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f92ac9-1086-4b6c-a4d1-1774bf9c4782 None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:53:12.372614 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:53:12.395490 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5f9e50-0d00-4b86-9802-263e46ec68bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:53:12.396099 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5f9e50-0d00-4b86-9802-263e46ec68bb None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:53:12.396376 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5f9e50-0d00-4b86-9802-263e46ec68bb None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:53:12.396647 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5f9e50-0d00-4b86-9802-263e46ec68bb None admin] PATH_INFO: `/v3/domains` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:53:12.397283 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c5f9e50-0d00-4b86-9802-263e46ec68bb None admin] RBAC: Adding query filter params (name=Default) {{(pid=81423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 10:53:12.397581 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c5f9e50-0d00-4b86-9802-263e46ec68bb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:53:12.399099 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c5f9e50-0d00-4b86-9802-263e46ec68bb None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:53:12.416451 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 528/1065] 10.222.0.40 () {56 vars in 1109 bytes} [Fri Aug 26 10:53:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 10:53:12.417872 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2981945-3895-4c7a-b0e9-46c6e3f8307b None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:53:12.516957 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a236ecaa-4d8d-48ce-b2a2-58d4dd5999aa None None] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:53:12.517282 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a236ecaa-4d8d-48ce-b2a2-58d4dd5999aa None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:53:12.517500 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a236ecaa-4d8d-48ce-b2a2-58d4dd5999aa None None] PATH_INFO: `/` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:53:12.519147 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 529/1066] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 26 10:53:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 26 10:53:12.524741 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb6ee67-fbf8-43e4-9db4-7fff56af51a1 None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:53:12.524928 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2981945-3895-4c7a-b0e9-46c6e3f8307b None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:53:12.525047 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb6ee67-fbf8-43e4-9db4-7fff56af51a1 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:53:12.525277 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb6ee67-fbf8-43e4-9db4-7fff56af51a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:53:12.527544 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:53:12.545137 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-3fb6ee67-fbf8-43e4-9db4-7fff56af51a1 None None] MFA Rules not processed for user `ca155a48e2024c32af3fd2e19e5fac11`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 10:53:12.546353 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-3fb6ee67-fbf8-43e4-9db4-7fff56af51a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 10:53:12.570704 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6875c6-0289-4752-9a24-329d2720e198 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:53:12.571657 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6875c6-0289-4752-9a24-329d2720e198 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:53:12.572231 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6875c6-0289-4752-9a24-329d2720e198 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:53:12.572550 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6875c6-0289-4752-9a24-329d2720e198 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:53:12.581153 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd6875c6-0289-4752-9a24-329d2720e198 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:53:12.582659 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd6875c6-0289-4752-9a24-329d2720e198 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:53:12.596321 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application [None req-cd6875c6-0289-4752-9a24-329d2720e198 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 26 10:53:12.596321 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 10:53:12.596321 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 10:53:12.596321 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 10:53:12.596321 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 10:53:12.596321 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 10:53:12.596321 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 10:53:12.596321 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 10:53:12.596321 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 10:53:12.596321 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 10:53:12.596321 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 10:53:12.596321 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 10:53:12.596321 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 26 10:53:12.596321 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 26 10:53:12.596321 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 26 10:53:12.596321 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 26 10:53:12.597337 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 10:53:12.597337 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 10:53:12.597337 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 26 10:53:12.597337 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 10:53:12.597337 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 26 10:53:12.597337 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 10:53:12.597337 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 10:53:12.597337 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 10:53:12.597337 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 10:53:12.597337 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 10:53:12.597337 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 10:53:12.597337 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application with Lock( Aug 26 10:53:12.597337 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 10:53:12.597337 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._enter() Aug 26 10:53:12.597337 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 10:53:12.597337 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 10:53:12.597337 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 10:53:12.597337 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.creator() Aug 26 10:53:12.598403 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 10:53:12.598403 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application created_value = creator( Aug 26 10:53:12.598403 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 26 10:53:12.598403 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 26 10:53:12.598403 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 26 10:53:12.598403 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 26 10:53:12.598403 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 26 10:53:12.598403 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 26 10:53:12.598403 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 26 10:53:12.598403 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Aug 26 10:53:12.598403 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 538/1067] 10.222.0.40 () {56 vars in 1093 bytes} [Fri Aug 26 10:53:12 2022] GET /identity/v3/users/nova => generated 82 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 10:53:12.603190 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6875c6-0289-4752-9a24-329d2720e198 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:53:12.620613 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 530/1068] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 26 10:53:12 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 26 10:53:12.707033 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6875c6-0289-4752-9a24-329d2720e198 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:53:12.741127 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee119a3-3222-4fbe-9d20-c04c166855b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:53:12.741735 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee119a3-3222-4fbe-9d20-c04c166855b7 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:53:12.742040 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee119a3-3222-4fbe-9d20-c04c166855b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:53:12.742319 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee119a3-3222-4fbe-9d20-c04c166855b7 None admin] PATH_INFO: `/v3/users` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:53:12.743039 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee119a3-3222-4fbe-9d20-c04c166855b7 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 10:53:12.743343 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee119a3-3222-4fbe-9d20-c04c166855b7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:53:12.744767 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee119a3-3222-4fbe-9d20-c04c166855b7 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:53:12.762073 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 539/1069] 10.222.0.40 () {56 vars in 1133 bytes} [Fri Aug 26 10:53:12 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 10:53:12.768290 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb6ee67-fbf8-43e4-9db4-7fff56af51a1 None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:53:12.884041 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb6ee67-fbf8-43e4-9db4-7fff56af51a1 None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:53:12.885716 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:53:12.891949 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee119a3-3222-4fbe-9d20-c04c166855b7 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:53:12.917747 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:53:12.922338 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eebf3c2-3eba-4d78-a994-a6b2dc9f191a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:53:12.923152 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eebf3c2-3eba-4d78-a994-a6b2dc9f191a None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:53:12.923440 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eebf3c2-3eba-4d78-a994-a6b2dc9f191a None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:53:12.923718 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eebf3c2-3eba-4d78-a994-a6b2dc9f191a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:53:12.934194 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eebf3c2-3eba-4d78-a994-a6b2dc9f191a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:53:12.935644 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eebf3c2-3eba-4d78-a994-a6b2dc9f191a None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:53:12.952474 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application [None req-4eebf3c2-3eba-4d78-a994-a6b2dc9f191a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 10:53:12.952474 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 10:53:12.952474 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 10:53:12.952474 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 10:53:12.952474 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 10:53:12.952474 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 10:53:12.952474 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 10:53:12.952474 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 10:53:12.952474 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 10:53:12.952474 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 10:53:12.952474 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 10:53:12.952474 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 10:53:12.952474 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 10:53:12.952474 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 10:53:12.952474 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 10:53:12.952474 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 10:53:12.954019 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 10:53:12.954019 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 10:53:12.954019 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 10:53:12.954019 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 10:53:12.954019 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 10:53:12.954019 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 10:53:12.954019 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 10:53:12.954019 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application with Lock( Aug 26 10:53:12.954019 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 10:53:12.954019 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._enter() Aug 26 10:53:12.954019 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 10:53:12.954019 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 10:53:12.954019 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 10:53:12.954019 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.creator() Aug 26 10:53:12.954019 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 10:53:12.954019 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application created_value = creator( Aug 26 10:53:12.954019 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 10:53:12.954986 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 10:53:12.954986 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 10:53:12.954986 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Aug 26 10:53:12.958332 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 531/1070] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Aug 26 10:53:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 10:53:12.960577 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eebf3c2-3eba-4d78-a994-a6b2dc9f191a None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:53:13.004918 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee119a3-3222-4fbe-9d20-c04c166855b7 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:53:13.043798 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2ed400-705f-402f-b6be-616e8cb0c91f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:53:13.044568 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2ed400-705f-402f-b6be-616e8cb0c91f None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:53:13.044849 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2ed400-705f-402f-b6be-616e8cb0c91f None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:53:13.045123 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2ed400-705f-402f-b6be-616e8cb0c91f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:53:13.069143 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eebf3c2-3eba-4d78-a994-a6b2dc9f191a None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:53:13.108252 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357bb129-5ea3-4a44-aa01-306762548fbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:53:13.109390 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357bb129-5ea3-4a44-aa01-306762548fbe None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:53:13.109390 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357bb129-5ea3-4a44-aa01-306762548fbe None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:53:13.109676 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357bb129-5ea3-4a44-aa01-306762548fbe None admin] PATH_INFO: `/v3/domains` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:53:13.110563 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-357bb129-5ea3-4a44-aa01-306762548fbe None admin] RBAC: Adding query filter params (name=Default) {{(pid=81423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 10:53:13.110859 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-357bb129-5ea3-4a44-aa01-306762548fbe None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:53:13.112131 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-357bb129-5ea3-4a44-aa01-306762548fbe None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:53:13.117082 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d2ed400-705f-402f-b6be-616e8cb0c91f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:53:13.118555 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d2ed400-705f-402f-b6be-616e8cb0c91f None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:53:13.130217 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 532/1071] 10.222.0.40 () {56 vars in 1109 bytes} [Fri Aug 26 10:53:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 10:53:13.131179 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357bb129-5ea3-4a44-aa01-306762548fbe None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:53:13.148269 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:53:13.151539 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:53:13.235286 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 540/1072] 10.222.0.40 () {58 vars in 1282 bytes} [Fri Aug 26 10:53:12 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 26 10:53:13.240417 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357bb129-5ea3-4a44-aa01-306762548fbe None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:53:13.241085 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2ed400-705f-402f-b6be-616e8cb0c91f None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:53:13.276654 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124ffaa9-4550-4bfe-bfcd-685accaa06d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:53:13.277294 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124ffaa9-4550-4bfe-bfcd-685accaa06d5 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:53:13.277569 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124ffaa9-4550-4bfe-bfcd-685accaa06d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:53:13.277842 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124ffaa9-4550-4bfe-bfcd-685accaa06d5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:53:13.284769 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-124ffaa9-4550-4bfe-bfcd-685accaa06d5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:53:13.286221 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-124ffaa9-4550-4bfe-bfcd-685accaa06d5 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:53:13.298671 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application [None req-124ffaa9-4550-4bfe-bfcd-685accaa06d5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 10:53:13.298671 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 10:53:13.298671 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 10:53:13.298671 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 10:53:13.298671 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 10:53:13.298671 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 10:53:13.298671 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 10:53:13.298671 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 10:53:13.298671 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 10:53:13.298671 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 10:53:13.298671 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 10:53:13.298671 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 10:53:13.298671 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 26 10:53:13.298671 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 26 10:53:13.298671 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 26 10:53:13.299637 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 26 10:53:13.299637 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 10:53:13.299637 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 10:53:13.299637 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 10:53:13.299637 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 10:53:13.299637 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 10:53:13.299637 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 10:53:13.299637 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 10:53:13.299637 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application with Lock( Aug 26 10:53:13.299637 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 10:53:13.299637 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._enter() Aug 26 10:53:13.299637 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 10:53:13.299637 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 10:53:13.299637 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 10:53:13.299637 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.creator() Aug 26 10:53:13.299637 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 10:53:13.299637 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application created_value = creator( Aug 26 10:53:13.301110 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 26 10:53:13.301110 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 26 10:53:13.301110 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 26 10:53:13.301110 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 26 10:53:13.301110 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 26 10:53:13.301110 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 26 10:53:13.301110 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 10:53:13.301110 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Aug 26 10:53:13.301110 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 533/1073] 10.222.0.40 () {56 vars in 1111 bytes} [Fri Aug 26 10:53:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 26 10:53:13.305381 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124ffaa9-4550-4bfe-bfcd-685accaa06d5 None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:53:13.358561 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2ed400-705f-402f-b6be-616e8cb0c91f None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:53:13.383022 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:53:13.385160 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:53:13.395312 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551ee931-0844-4562-8269-f86d3167394d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:53:13.395941 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551ee931-0844-4562-8269-f86d3167394d None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:53:13.396212 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551ee931-0844-4562-8269-f86d3167394d None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:53:13.396486 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551ee931-0844-4562-8269-f86d3167394d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:53:13.406963 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-551ee931-0844-4562-8269-f86d3167394d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:53:13.408409 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-551ee931-0844-4562-8269-f86d3167394d None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:53:13.412053 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124ffaa9-4550-4bfe-bfcd-685accaa06d5 None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:53:13.423167 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application [None req-551ee931-0844-4562-8269-f86d3167394d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 10:53:13.423167 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 10:53:13.423167 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 10:53:13.423167 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 10:53:13.423167 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 10:53:13.423167 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 10:53:13.423167 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 10:53:13.423167 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 10:53:13.423167 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 10:53:13.423167 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 10:53:13.423167 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 10:53:13.423167 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 10:53:13.423167 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 10:53:13.423167 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 10:53:13.423167 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 10:53:13.423167 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 10:53:13.424161 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 10:53:13.424161 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 10:53:13.424161 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 10:53:13.424161 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 10:53:13.424161 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 10:53:13.424161 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 10:53:13.424161 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 10:53:13.424161 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application with Lock( Aug 26 10:53:13.424161 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 10:53:13.424161 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._enter() Aug 26 10:53:13.424161 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 10:53:13.424161 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 10:53:13.424161 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 10:53:13.424161 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.creator() Aug 26 10:53:13.424161 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 10:53:13.424161 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application created_value = creator( Aug 26 10:53:13.424161 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 10:53:13.425067 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 10:53:13.425067 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 10:53:13.425067 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Aug 26 10:53:13.425067 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 541/1074] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Aug 26 10:53:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 26 10:53:13.425556 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5208f7-5d57-4f8e-bdde-b8060c1602ae None None] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:53:13.425842 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5208f7-5d57-4f8e-bdde-b8060c1602ae None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:53:13.426149 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5208f7-5d57-4f8e-bdde-b8060c1602ae None None] PATH_INFO: `/` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:53:13.428392 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 542/1075] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 26 10:53:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 10:53:13.431005 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5208f7-5d57-4f8e-bdde-b8060c1602ae None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:53:13.450178 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4db5f7c-137e-419d-a576-84cbcbdda926 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:53:13.450784 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4db5f7c-137e-419d-a576-84cbcbdda926 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:53:13.451056 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4db5f7c-137e-419d-a576-84cbcbdda926 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:53:13.451325 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4db5f7c-137e-419d-a576-84cbcbdda926 None admin] PATH_INFO: `/v3/projects` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:53:13.452002 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4db5f7c-137e-419d-a576-84cbcbdda926 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 10:53:13.452297 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4db5f7c-137e-419d-a576-84cbcbdda926 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:53:13.454304 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4db5f7c-137e-419d-a576-84cbcbdda926 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:53:13.466711 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 534/1076] 10.222.0.40 () {56 vars in 1148 bytes} [Fri Aug 26 10:53:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 10:53:13.467872 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b3f3d9-0868-416a-8900-bfee6ae5b56c None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:53:13.468149 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b3f3d9-0868-416a-8900-bfee6ae5b56c None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:53:13.468419 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b3f3d9-0868-416a-8900-bfee6ae5b56c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:53:13.486847 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-e3b3f3d9-0868-416a-8900-bfee6ae5b56c None None] MFA Rules not processed for user `ca155a48e2024c32af3fd2e19e5fac11`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 10:53:13.488024 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-e3b3f3d9-0868-416a-8900-bfee6ae5b56c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 10:53:13.538802 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5208f7-5d57-4f8e-bdde-b8060c1602ae None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:53:13.555458 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 535/1077] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 26 10:53:13 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 26 10:53:13.556254 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b3f3d9-0868-416a-8900-bfee6ae5b56c None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:53:13.580193 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71200771-9b45-4979-b417-e01e0665bfa1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:53:13.580798 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71200771-9b45-4979-b417-e01e0665bfa1 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:53:13.581076 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71200771-9b45-4979-b417-e01e0665bfa1 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:53:13.581346 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71200771-9b45-4979-b417-e01e0665bfa1 None admin] PATH_INFO: `/v3/domains` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:53:13.582004 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71200771-9b45-4979-b417-e01e0665bfa1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 10:53:13.582334 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71200771-9b45-4979-b417-e01e0665bfa1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:53:13.583697 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71200771-9b45-4979-b417-e01e0665bfa1 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:53:13.596211 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 543/1078] 10.222.0.40 () {56 vars in 1109 bytes} [Fri Aug 26 10:53:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 10:53:13.634329 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:53:13.678357 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b3f3d9-0868-416a-8900-bfee6ae5b56c None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:53:13.716390 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dca3bcf-4e88-4f8e-9486-f6de67f6a2e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:53:13.717054 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dca3bcf-4e88-4f8e-9486-f6de67f6a2e6 None admin] REQUEST_METHOD: `PUT` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:53:13.717344 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dca3bcf-4e88-4f8e-9486-f6de67f6a2e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:53:13.717588 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dca3bcf-4e88-4f8e-9486-f6de67f6a2e6 None admin] PATH_INFO: `/v3/projects/0e30ccf0c1204727b786876c141dbcd2/users/1f4b4a662db049ecacc865ecff10bdb0/roles/3f7d20781437466aabcbf11bda248f3d` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:53:13.720237 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dca3bcf-4e88-4f8e-9486-f6de67f6a2e6 None admin] RBAC: Authorizing `identity:create_grant(project_id=0e30ccf0c1204727b786876c141dbcd2, user_id=1f4b4a662db049ecacc865ecff10bdb0, role_id=3f7d20781437466aabcbf11bda248f3d)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:53:13.721702 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dca3bcf-4e88-4f8e-9486-f6de67f6a2e6 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:53:13.737908 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 536/1079] 10.222.0.40 () {58 vars in 1439 bytes} [Fri Aug 26 10:53:13 2022] PUT /identity/v3/projects/0e30ccf0c1204727b786876c141dbcd2/users/1f4b4a662db049ecacc865ecff10bdb0/roles/3f7d20781437466aabcbf11bda248f3d => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 10:53:13.826753 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71200771-9b45-4979-b417-e01e0665bfa1 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:53:13.871025 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:53:13.946796 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71200771-9b45-4979-b417-e01e0665bfa1 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:53:13.980118 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5a0f8f-29c7-45e2-82ba-1cb965196c27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:53:13.980745 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5a0f8f-29c7-45e2-82ba-1cb965196c27 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:53:13.981018 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5a0f8f-29c7-45e2-82ba-1cb965196c27 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:53:13.981282 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5a0f8f-29c7-45e2-82ba-1cb965196c27 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:53:13.992928 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd5a0f8f-29c7-45e2-82ba-1cb965196c27 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 10:53:13.997545 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application [None req-dd5a0f8f-29c7-45e2-82ba-1cb965196c27 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 26 10:53:13.997545 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 10:53:13.997545 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 10:53:13.997545 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 10:53:13.997545 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 10:53:13.997545 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 10:53:13.997545 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 10:53:13.997545 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 10:53:13.997545 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 10:53:13.997545 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 10:53:13.997545 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 10:53:13.997545 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 10:53:13.997545 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 10:53:13.997545 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 10:53:13.997545 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 10:53:13.997545 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 10:53:13.998601 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 10:53:13.998601 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 10:53:13.998601 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 10:53:13.998601 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 10:53:13.998601 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 10:53:13.998601 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 10:53:13.998601 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 10:53:13.998601 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 10:53:13.998601 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 10:53:13.998601 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 10:53:13.998601 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 10:53:13.998601 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application with Lock( Aug 26 10:53:13.998601 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 10:53:13.998601 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._enter() Aug 26 10:53:13.998601 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 10:53:13.998601 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 10:53:13.999466 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 10:53:13.999466 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.creator() Aug 26 10:53:13.999466 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 10:53:13.999466 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application created_value = creator( Aug 26 10:53:13.999466 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 10:53:13.999466 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 10:53:13.999466 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 10:53:13.999466 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 10:53:13.999466 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 10:53:13.999466 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 10:53:13.999466 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 26 10:53:13.999466 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Aug 26 10:53:13.999466 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 544/1080] 10.222.0.40 () {56 vars in 1096 bytes} [Fri Aug 26 10:53:13 2022] GET /identity/v3/roles/admin => generated 83 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 10:53:14.005401 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dca3bcf-4e88-4f8e-9486-f6de67f6a2e6 None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:53:14.105060 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dca3bcf-4e88-4f8e-9486-f6de67f6a2e6 None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:53:14.129456 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:53:14.141429 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4d64b0-fcf4-48b3-aaa0-2932e8cfa5d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:53:14.142073 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4d64b0-fcf4-48b3-aaa0-2932e8cfa5d9 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:53:14.142349 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4d64b0-fcf4-48b3-aaa0-2932e8cfa5d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:53:14.142614 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4d64b0-fcf4-48b3-aaa0-2932e8cfa5d9 None admin] PATH_INFO: `/v3/roles` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:53:14.143273 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba4d64b0-fcf4-48b3-aaa0-2932e8cfa5d9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 10:53:14.143574 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba4d64b0-fcf4-48b3-aaa0-2932e8cfa5d9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:53:14.144906 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba4d64b0-fcf4-48b3-aaa0-2932e8cfa5d9 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:53:14.153389 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 537/1081] 10.222.0.40 () {56 vars in 1099 bytes} [Fri Aug 26 10:53:14 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 10:53:14.159045 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5a0f8f-29c7-45e2-82ba-1cb965196c27 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:53:14.252453 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:53:14.259280 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5a0f8f-29c7-45e2-82ba-1cb965196c27 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:53:14.292397 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1eeb1fd-f3a6-44f0-a16d-8923ea96eb1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:53:14.293024 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1eeb1fd-f3a6-44f0-a16d-8923ea96eb1f None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:53:14.293303 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1eeb1fd-f3a6-44f0-a16d-8923ea96eb1f None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:53:14.293578 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1eeb1fd-f3a6-44f0-a16d-8923ea96eb1f None admin] PATH_INFO: `/v3/users/76e608b7f48c4fe6a152d759ac0f648f` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:53:14.294723 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1eeb1fd-f3a6-44f0-a16d-8923ea96eb1f None admin] RBAC: Authorizing `identity:get_user(user_id=76e608b7f48c4fe6a152d759ac0f648f)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:53:14.296157 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1eeb1fd-f3a6-44f0-a16d-8923ea96eb1f None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:53:14.298362 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 545/1082] 10.222.0.40 () {56 vars in 1177 bytes} [Fri Aug 26 10:53:14 2022] GET /identity/v3/users/76e608b7f48c4fe6a152d759ac0f648f => generated 285 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 10:53:14.304054 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4d64b0-fcf4-48b3-aaa0-2932e8cfa5d9 None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:53:14.401599 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4d64b0-fcf4-48b3-aaa0-2932e8cfa5d9 None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:53:14.435664 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866382ad-7be2-4512-b244-ab6dedebef71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:53:14.436276 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866382ad-7be2-4512-b244-ab6dedebef71 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:53:14.436548 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866382ad-7be2-4512-b244-ab6dedebef71 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:53:14.436814 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866382ad-7be2-4512-b244-ab6dedebef71 None admin] PATH_INFO: `/v3/projects/647d6df216314e948b277a186b28a0ac` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:53:14.438486 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-866382ad-7be2-4512-b244-ab6dedebef71 None admin] RBAC: Authorizing `identity:get_project(project_id=647d6df216314e948b277a186b28a0ac)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:53:14.439939 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-866382ad-7be2-4512-b244-ab6dedebef71 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:53:14.441573 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 538/1083] 10.222.0.40 () {56 vars in 1186 bytes} [Fri Aug 26 10:53:14 2022] GET /identity/v3/projects/647d6df216314e948b277a186b28a0ac => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 10:53:14.447424 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1eeb1fd-f3a6-44f0-a16d-8923ea96eb1f None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:53:14.543593 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1eeb1fd-f3a6-44f0-a16d-8923ea96eb1f None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:53:14.575336 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e98e46b-a60c-4de0-9b3a-d91d6ab53c8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:53:14.575982 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e98e46b-a60c-4de0-9b3a-d91d6ab53c8d None admin] REQUEST_METHOD: `PUT` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:53:14.576254 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e98e46b-a60c-4de0-9b3a-d91d6ab53c8d None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:53:14.576522 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e98e46b-a60c-4de0-9b3a-d91d6ab53c8d None admin] PATH_INFO: `/v3/projects/647d6df216314e948b277a186b28a0ac/users/76e608b7f48c4fe6a152d759ac0f648f/roles/1f97e4602e2e4d6498913a81f9ae8d6a` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:53:14.578987 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e98e46b-a60c-4de0-9b3a-d91d6ab53c8d None admin] RBAC: Authorizing `identity:create_grant(project_id=647d6df216314e948b277a186b28a0ac, user_id=76e608b7f48c4fe6a152d759ac0f648f, role_id=1f97e4602e2e4d6498913a81f9ae8d6a)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:53:14.580395 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e98e46b-a60c-4de0-9b3a-d91d6ab53c8d None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:53:14.592959 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 546/1084] 10.222.0.40 () {58 vars in 1439 bytes} [Fri Aug 26 10:53:14 2022] PUT /identity/v3/projects/647d6df216314e948b277a186b28a0ac/users/76e608b7f48c4fe6a152d759ac0f648f/roles/1f97e4602e2e4d6498913a81f9ae8d6a => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 10:53:15.658882 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac49ff3-1f9d-489c-9399-1d7c6d0a852b None None] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:53:15.659413 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac49ff3-1f9d-489c-9399-1d7c6d0a852b None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:53:15.659413 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac49ff3-1f9d-489c-9399-1d7c6d0a852b None None] PATH_INFO: `/` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:53:15.660765 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 539/1085] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 26 10:53:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 10:53:15.666058 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91cdf83c-1ff2-41a3-b5bf-231c748f823d None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:53:15.666437 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91cdf83c-1ff2-41a3-b5bf-231c748f823d None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:53:15.666715 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91cdf83c-1ff2-41a3-b5bf-231c748f823d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:53:15.686228 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-91cdf83c-1ff2-41a3-b5bf-231c748f823d None None] MFA Rules not processed for user `ca155a48e2024c32af3fd2e19e5fac11`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 10:53:15.688042 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-91cdf83c-1ff2-41a3-b5bf-231c748f823d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 10:53:15.704740 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:53:15.754387 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 547/1086] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 26 10:53:15 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 26 10:53:16.021852 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac49ff3-1f9d-489c-9399-1d7c6d0a852b None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:53:16.077912 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:53:16.123854 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac49ff3-1f9d-489c-9399-1d7c6d0a852b None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:53:16.157439 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e718fb-c1b6-4d95-90b9-0c07d39f98d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:53:16.158082 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e718fb-c1b6-4d95-90b9-0c07d39f98d0 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:53:16.158354 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e718fb-c1b6-4d95-90b9-0c07d39f98d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:53:16.158616 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e718fb-c1b6-4d95-90b9-0c07d39f98d0 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:53:16.168415 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e718fb-c1b6-4d95-90b9-0c07d39f98d0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 10:53:16.173825 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application [None req-87e718fb-c1b6-4d95-90b9-0c07d39f98d0 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 26 10:53:16.173825 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 10:53:16.173825 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 10:53:16.173825 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 10:53:16.173825 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 10:53:16.173825 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 10:53:16.173825 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 10:53:16.173825 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 10:53:16.173825 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 10:53:16.173825 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 10:53:16.173825 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 10:53:16.173825 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 10:53:16.173825 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 10:53:16.173825 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 10:53:16.173825 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 10:53:16.173825 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 10:53:16.174907 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 10:53:16.174907 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 10:53:16.174907 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 10:53:16.174907 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 10:53:16.174907 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 10:53:16.174907 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 10:53:16.174907 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 10:53:16.174907 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 10:53:16.174907 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 10:53:16.174907 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 10:53:16.174907 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 10:53:16.174907 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application with Lock( Aug 26 10:53:16.174907 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 10:53:16.174907 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._enter() Aug 26 10:53:16.174907 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 10:53:16.174907 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 10:53:16.175744 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 10:53:16.175744 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.creator() Aug 26 10:53:16.175744 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 10:53:16.175744 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application created_value = creator( Aug 26 10:53:16.175744 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 10:53:16.175744 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 10:53:16.175744 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 10:53:16.175744 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 10:53:16.175744 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 10:53:16.175744 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 10:53:16.175744 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 26 10:53:16.175744 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Aug 26 10:53:16.175744 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 540/1087] 10.222.0.40 () {56 vars in 1120 bytes} [Fri Aug 26 10:53:16 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 10:53:16.180838 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91cdf83c-1ff2-41a3-b5bf-231c748f823d None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:53:16.279497 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91cdf83c-1ff2-41a3-b5bf-231c748f823d None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:53:16.313423 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8a9c3f-d7e1-40f3-86fa-66d794f9002d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:53:16.314048 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8a9c3f-d7e1-40f3-86fa-66d794f9002d None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:53:16.314327 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8a9c3f-d7e1-40f3-86fa-66d794f9002d None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:53:16.314596 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8a9c3f-d7e1-40f3-86fa-66d794f9002d None admin] PATH_INFO: `/v3/roles` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:53:16.315251 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb8a9c3f-d7e1-40f3-86fa-66d794f9002d None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 10:53:16.315549 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb8a9c3f-d7e1-40f3-86fa-66d794f9002d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:53:16.316906 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb8a9c3f-d7e1-40f3-86fa-66d794f9002d None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:53:16.324558 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 548/1088] 10.222.0.40 () {56 vars in 1115 bytes} [Fri Aug 26 10:53:16 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 10:53:16.330752 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e718fb-c1b6-4d95-90b9-0c07d39f98d0 None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:53:16.429444 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e718fb-c1b6-4d95-90b9-0c07d39f98d0 None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:53:16.443897 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:53:16.464609 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5714e560-41e8-465f-beb1-e32ad86f1782 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:53:16.465234 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5714e560-41e8-465f-beb1-e32ad86f1782 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:53:16.465509 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5714e560-41e8-465f-beb1-e32ad86f1782 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:53:16.465777 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5714e560-41e8-465f-beb1-e32ad86f1782 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:53:16.475951 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5714e560-41e8-465f-beb1-e32ad86f1782 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:53:16.477379 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5714e560-41e8-465f-beb1-e32ad86f1782 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:53:16.492830 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application [None req-5714e560-41e8-465f-beb1-e32ad86f1782 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 10:53:16.492830 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 10:53:16.492830 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 10:53:16.492830 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 10:53:16.492830 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 10:53:16.492830 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 10:53:16.492830 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 10:53:16.492830 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 10:53:16.492830 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 10:53:16.492830 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 10:53:16.492830 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 10:53:16.492830 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 10:53:16.492830 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 10:53:16.492830 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 10:53:16.492830 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 10:53:16.492830 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 10:53:16.494011 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 10:53:16.494011 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 10:53:16.494011 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 10:53:16.494011 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 10:53:16.494011 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 10:53:16.494011 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 10:53:16.494011 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 10:53:16.494011 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application with Lock( Aug 26 10:53:16.494011 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 10:53:16.494011 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._enter() Aug 26 10:53:16.494011 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 10:53:16.494011 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 10:53:16.494011 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 10:53:16.494011 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.creator() Aug 26 10:53:16.494011 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 10:53:16.494011 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application created_value = creator( Aug 26 10:53:16.494011 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 10:53:16.494923 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 10:53:16.494923 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 10:53:16.494923 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Aug 26 10:53:16.494923 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 541/1089] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Aug 26 10:53:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 26 10:53:16.499742 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8a9c3f-d7e1-40f3-86fa-66d794f9002d None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:53:16.516458 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:53:16.528837 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5992c769-a2a0-4e1f-974b-7cf4009a5ef0 None None] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:53:16.529111 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5992c769-a2a0-4e1f-974b-7cf4009a5ef0 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:53:16.529388 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5992c769-a2a0-4e1f-974b-7cf4009a5ef0 None None] PATH_INFO: `/` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:53:16.530813 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 542/1090] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 26 10:53:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 26 10:53:16.536457 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed94e599-8ad7-458c-848e-3e4c1ea826c8 None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:53:16.536732 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed94e599-8ad7-458c-848e-3e4c1ea826c8 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:53:16.536998 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed94e599-8ad7-458c-848e-3e4c1ea826c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:53:16.555181 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-ed94e599-8ad7-458c-848e-3e4c1ea826c8 None None] MFA Rules not processed for user `ca155a48e2024c32af3fd2e19e5fac11`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 10:53:16.556352 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-ed94e599-8ad7-458c-848e-3e4c1ea826c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 10:53:16.601945 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8a9c3f-d7e1-40f3-86fa-66d794f9002d None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:53:16.638012 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 543/1091] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 26 10:53:16 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 26 10:53:16.638231 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676ae2d1-84d3-4976-8021-34764f901f40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:53:16.638559 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676ae2d1-84d3-4976-8021-34764f901f40 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:53:16.638834 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676ae2d1-84d3-4976-8021-34764f901f40 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:53:16.639100 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676ae2d1-84d3-4976-8021-34764f901f40 None admin] PATH_INFO: `/v3/domains` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:53:16.639733 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-676ae2d1-84d3-4976-8021-34764f901f40 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 10:53:16.640033 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-676ae2d1-84d3-4976-8021-34764f901f40 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:53:16.641488 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-676ae2d1-84d3-4976-8021-34764f901f40 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:53:16.653158 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 549/1092] 10.222.0.40 () {56 vars in 1109 bytes} [Fri Aug 26 10:53:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 10:53:16.659240 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed94e599-8ad7-458c-848e-3e4c1ea826c8 None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:53:16.750053 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:53:16.758049 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed94e599-8ad7-458c-848e-3e4c1ea826c8 None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:53:16.791530 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8a59ec-b736-44ee-a34a-36864a7273b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:53:16.792080 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8a59ec-b736-44ee-a34a-36864a7273b4 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:53:16.792354 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8a59ec-b736-44ee-a34a-36864a7273b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:53:16.792620 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8a59ec-b736-44ee-a34a-36864a7273b4 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:53:16.800346 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d8a59ec-b736-44ee-a34a-36864a7273b4 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:53:16.801755 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d8a59ec-b736-44ee-a34a-36864a7273b4 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:53:16.813186 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application [None req-8d8a59ec-b736-44ee-a34a-36864a7273b4 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 26 10:53:16.813186 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 10:53:16.813186 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 10:53:16.813186 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 10:53:16.813186 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 10:53:16.813186 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 10:53:16.813186 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 10:53:16.813186 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 10:53:16.813186 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 10:53:16.813186 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 10:53:16.813186 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 10:53:16.813186 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 10:53:16.813186 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 26 10:53:16.813186 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 26 10:53:16.813186 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 26 10:53:16.813186 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 26 10:53:16.814308 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 10:53:16.814308 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 10:53:16.814308 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 26 10:53:16.814308 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 10:53:16.814308 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 26 10:53:16.814308 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 10:53:16.814308 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 10:53:16.814308 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 10:53:16.814308 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 10:53:16.814308 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 10:53:16.814308 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 10:53:16.814308 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application with Lock( Aug 26 10:53:16.814308 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 10:53:16.814308 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._enter() Aug 26 10:53:16.814308 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 10:53:16.814308 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 10:53:16.814308 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 10:53:16.814308 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.creator() Aug 26 10:53:16.815280 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 10:53:16.815280 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application created_value = creator( Aug 26 10:53:16.815280 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 26 10:53:16.815280 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 26 10:53:16.815280 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 26 10:53:16.815280 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 26 10:53:16.815280 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 26 10:53:16.815280 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 26 10:53:16.815280 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 26 10:53:16.815280 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Aug 26 10:53:16.815280 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 544/1093] 10.222.0.40 () {56 vars in 1093 bytes} [Fri Aug 26 10:53:16 2022] GET /identity/v3/users/nova => generated 82 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 10:53:16.819973 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676ae2d1-84d3-4976-8021-34764f901f40 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:53:16.897416 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:53:16.898009 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8a59ec-b736-44ee-a34a-36864a7273b4 None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:53:16.924020 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676ae2d1-84d3-4976-8021-34764f901f40 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:53:16.957676 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb22191-d98d-4679-bb17-6c06fc47d46b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:53:16.958310 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb22191-d98d-4679-bb17-6c06fc47d46b None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:53:16.958589 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb22191-d98d-4679-bb17-6c06fc47d46b None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:53:16.958860 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb22191-d98d-4679-bb17-6c06fc47d46b None admin] PATH_INFO: `/v3/users` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:53:16.959576 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb22191-d98d-4679-bb17-6c06fc47d46b None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 10:53:16.959875 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb22191-d98d-4679-bb17-6c06fc47d46b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:53:16.961308 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb22191-d98d-4679-bb17-6c06fc47d46b None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:53:16.978698 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 550/1094] 10.222.0.40 () {56 vars in 1133 bytes} [Fri Aug 26 10:53:16 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 10:53:16.984838 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb22191-d98d-4679-bb17-6c06fc47d46b None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:53:17.001072 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8a59ec-b736-44ee-a34a-36864a7273b4 None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:53:17.035352 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38c1d26-7139-4687-acb6-7074b0280033 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:53:17.035978 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38c1d26-7139-4687-acb6-7074b0280033 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:53:17.036259 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38c1d26-7139-4687-acb6-7074b0280033 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:53:17.036526 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38c1d26-7139-4687-acb6-7074b0280033 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:53:17.047135 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c38c1d26-7139-4687-acb6-7074b0280033 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 10:53:17.051788 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application [None req-c38c1d26-7139-4687-acb6-7074b0280033 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 26 10:53:17.051788 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 10:53:17.051788 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 10:53:17.051788 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 10:53:17.051788 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 10:53:17.051788 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 10:53:17.051788 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 10:53:17.051788 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 10:53:17.051788 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 10:53:17.051788 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 10:53:17.051788 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 10:53:17.051788 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 10:53:17.051788 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 10:53:17.051788 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 10:53:17.051788 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 10:53:17.051788 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 10:53:17.052729 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 10:53:17.052729 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 10:53:17.052729 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 10:53:17.052729 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 10:53:17.052729 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 10:53:17.052729 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 10:53:17.052729 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 10:53:17.052729 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 10:53:17.052729 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 10:53:17.052729 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 10:53:17.052729 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 10:53:17.052729 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application with Lock( Aug 26 10:53:17.052729 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 10:53:17.052729 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._enter() Aug 26 10:53:17.052729 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 10:53:17.052729 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 10:53:17.053759 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 10:53:17.053759 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.creator() Aug 26 10:53:17.053759 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 10:53:17.053759 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application created_value = creator( Aug 26 10:53:17.053759 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 10:53:17.053759 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 10:53:17.053759 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 10:53:17.053759 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 10:53:17.053759 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 10:53:17.053759 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 10:53:17.053759 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 26 10:53:17.053759 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Aug 26 10:53:17.053759 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 545/1095] 10.222.0.40 () {56 vars in 1096 bytes} [Fri Aug 26 10:53:16 2022] GET /identity/v3/roles/admin => generated 83 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 26 10:53:17.058722 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38c1d26-7139-4687-acb6-7074b0280033 None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:53:17.076069 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:53:17.087580 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb22191-d98d-4679-bb17-6c06fc47d46b None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:53:17.117342 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:53:17.123240 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854d5e57-0cd9-4e96-8438-1909f5446e2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:53:17.123876 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854d5e57-0cd9-4e96-8438-1909f5446e2f None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:53:17.124159 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854d5e57-0cd9-4e96-8438-1909f5446e2f None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:53:17.124440 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854d5e57-0cd9-4e96-8438-1909f5446e2f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:53:17.158813 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38c1d26-7139-4687-acb6-7074b0280033 None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:53:17.191231 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-854d5e57-0cd9-4e96-8438-1909f5446e2f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:53:17.191405 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29fa4b81-7117-4cc0-847a-8624473d83da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:53:17.191900 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29fa4b81-7117-4cc0-847a-8624473d83da None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:53:17.192177 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29fa4b81-7117-4cc0-847a-8624473d83da None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:53:17.192450 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29fa4b81-7117-4cc0-847a-8624473d83da None admin] PATH_INFO: `/v3/roles` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:53:17.192676 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-854d5e57-0cd9-4e96-8438-1909f5446e2f None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:53:17.193111 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29fa4b81-7117-4cc0-847a-8624473d83da None admin] RBAC: Adding query filter params (name=admin) {{(pid=81423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 10:53:17.193412 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29fa4b81-7117-4cc0-847a-8624473d83da None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:53:17.194785 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29fa4b81-7117-4cc0-847a-8624473d83da None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:53:17.203879 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 546/1096] 10.222.0.40 () {56 vars in 1099 bytes} [Fri Aug 26 10:53:17 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 10:53:17.209616 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29fa4b81-7117-4cc0-847a-8624473d83da None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:53:17.286538 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:53:17.291563 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 551/1097] 10.222.0.40 () {58 vars in 1282 bytes} [Fri Aug 26 10:53:16 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 26 10:53:17.297343 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854d5e57-0cd9-4e96-8438-1909f5446e2f None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:53:17.308825 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29fa4b81-7117-4cc0-847a-8624473d83da None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:53:17.327156 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:53:17.344317 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e940b3bd-13bd-45c9-b912-773db15eba48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:53:17.344941 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e940b3bd-13bd-45c9-b912-773db15eba48 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:53:17.345222 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e940b3bd-13bd-45c9-b912-773db15eba48 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:53:17.345495 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e940b3bd-13bd-45c9-b912-773db15eba48 None admin] PATH_INFO: `/v3/users/76e608b7f48c4fe6a152d759ac0f648f` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:53:17.346708 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e940b3bd-13bd-45c9-b912-773db15eba48 None admin] RBAC: Authorizing `identity:get_user(user_id=76e608b7f48c4fe6a152d759ac0f648f)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:53:17.348149 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e940b3bd-13bd-45c9-b912-773db15eba48 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:53:17.349733 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 547/1098] 10.222.0.40 () {56 vars in 1177 bytes} [Fri Aug 26 10:53:17 2022] GET /identity/v3/users/76e608b7f48c4fe6a152d759ac0f648f => generated 285 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 10:53:17.355919 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e940b3bd-13bd-45c9-b912-773db15eba48 None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:53:17.396479 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854d5e57-0cd9-4e96-8438-1909f5446e2f None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:53:17.429754 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6338e8-d457-4f21-b2a1-994061b7aa55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:53:17.430405 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6338e8-d457-4f21-b2a1-994061b7aa55 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:53:17.430686 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6338e8-d457-4f21-b2a1-994061b7aa55 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:53:17.430958 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6338e8-d457-4f21-b2a1-994061b7aa55 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:53:17.441732 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c6338e8-d457-4f21-b2a1-994061b7aa55 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:53:17.443210 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c6338e8-d457-4f21-b2a1-994061b7aa55 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:53:17.458295 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application [None req-1c6338e8-d457-4f21-b2a1-994061b7aa55 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 10:53:17.458295 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 10:53:17.458295 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 10:53:17.458295 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 10:53:17.458295 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 10:53:17.458295 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 10:53:17.458295 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 10:53:17.458295 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 10:53:17.458295 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 10:53:17.458295 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 10:53:17.458295 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 10:53:17.458295 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 10:53:17.458295 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 10:53:17.458295 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 10:53:17.458295 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 10:53:17.458295 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 10:53:17.459448 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 10:53:17.459448 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 10:53:17.459448 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 10:53:17.459448 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 10:53:17.459448 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 10:53:17.459448 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 10:53:17.459448 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 10:53:17.459448 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application with Lock( Aug 26 10:53:17.459448 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 10:53:17.459448 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._enter() Aug 26 10:53:17.459448 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 10:53:17.459448 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 10:53:17.459448 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 10:53:17.459448 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.creator() Aug 26 10:53:17.459448 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 10:53:17.459448 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application created_value = creator( Aug 26 10:53:17.459448 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 10:53:17.460529 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 10:53:17.460529 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 10:53:17.460529 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Aug 26 10:53:17.460742 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e940b3bd-13bd-45c9-b912-773db15eba48 None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:53:17.462308 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 552/1099] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Aug 26 10:53:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 26 10:53:17.464884 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6338e8-d457-4f21-b2a1-994061b7aa55 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:53:17.492951 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d2b03c-c999-40e3-8972-26f3699db0cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:53:17.493550 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d2b03c-c999-40e3-8972-26f3699db0cd None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:53:17.493826 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d2b03c-c999-40e3-8972-26f3699db0cd None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:53:17.494132 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d2b03c-c999-40e3-8972-26f3699db0cd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:53:17.512008 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:53:17.547784 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:53:17.565002 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6338e8-d457-4f21-b2a1-994061b7aa55 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:53:17.565357 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d2b03c-c999-40e3-8972-26f3699db0cd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:53:17.566897 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d2b03c-c999-40e3-8972-26f3699db0cd None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:53:17.599127 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf82dad-02ee-4e2c-9524-2e99bef2648d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:53:17.599730 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf82dad-02ee-4e2c-9524-2e99bef2648d None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:53:17.600010 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf82dad-02ee-4e2c-9524-2e99bef2648d None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:53:17.600266 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf82dad-02ee-4e2c-9524-2e99bef2648d None admin] PATH_INFO: `/v3/domains` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:53:17.600899 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf82dad-02ee-4e2c-9524-2e99bef2648d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 10:53:17.601194 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf82dad-02ee-4e2c-9524-2e99bef2648d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:53:17.602588 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf82dad-02ee-4e2c-9524-2e99bef2648d None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:53:17.614333 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 553/1100] 10.222.0.40 () {56 vars in 1109 bytes} [Fri Aug 26 10:53:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 10:53:17.619801 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf82dad-02ee-4e2c-9524-2e99bef2648d None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:53:17.665997 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 548/1101] 10.222.0.40 () {58 vars in 1282 bytes} [Fri Aug 26 10:53:17 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 26 10:53:17.672305 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d2b03c-c999-40e3-8972-26f3699db0cd None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:53:17.716186 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:53:17.718616 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf82dad-02ee-4e2c-9524-2e99bef2648d None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:53:17.752758 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a016932d-9442-44f8-9cbd-057099dfae78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:53:17.753373 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a016932d-9442-44f8-9cbd-057099dfae78 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:53:17.753649 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a016932d-9442-44f8-9cbd-057099dfae78 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:53:17.753913 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a016932d-9442-44f8-9cbd-057099dfae78 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:53:17.760475 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a016932d-9442-44f8-9cbd-057099dfae78 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:53:17.761908 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a016932d-9442-44f8-9cbd-057099dfae78 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:53:17.770614 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d2b03c-c999-40e3-8972-26f3699db0cd None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:53:17.773477 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application [None req-a016932d-9442-44f8-9cbd-057099dfae78 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 10:53:17.773477 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 10:53:17.773477 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 10:53:17.773477 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 10:53:17.773477 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 10:53:17.773477 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 10:53:17.773477 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 10:53:17.773477 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 10:53:17.773477 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 10:53:17.773477 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 10:53:17.773477 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 10:53:17.773477 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 10:53:17.773477 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 26 10:53:17.773477 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 26 10:53:17.773477 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 26 10:53:17.774504 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 26 10:53:17.774504 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 10:53:17.774504 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 10:53:17.774504 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 10:53:17.774504 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 10:53:17.774504 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 10:53:17.774504 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 10:53:17.774504 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 10:53:17.774504 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application with Lock( Aug 26 10:53:17.774504 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 10:53:17.774504 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._enter() Aug 26 10:53:17.774504 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 10:53:17.774504 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 10:53:17.774504 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 10:53:17.774504 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.creator() Aug 26 10:53:17.774504 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 10:53:17.774504 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application created_value = creator( Aug 26 10:53:17.775409 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 26 10:53:17.775409 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 26 10:53:17.775409 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 26 10:53:17.775409 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 26 10:53:17.775409 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 26 10:53:17.775409 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 26 10:53:17.775409 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 10:53:17.775409 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Aug 26 10:53:17.775409 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 554/1102] 10.222.0.40 () {56 vars in 1111 bytes} [Fri Aug 26 10:53:17 2022] GET /identity/v3/projects/service => generated 88 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 26 10:53:17.780290 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a016932d-9442-44f8-9cbd-057099dfae78 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:53:17.784376 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:53:17.805154 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994d1f22-4f11-4a51-b3dc-cd921a5f3df1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:53:17.805707 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994d1f22-4f11-4a51-b3dc-cd921a5f3df1 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:53:17.806027 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994d1f22-4f11-4a51-b3dc-cd921a5f3df1 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:53:17.806285 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994d1f22-4f11-4a51-b3dc-cd921a5f3df1 None admin] PATH_INFO: `/v3/projects/647d6df216314e948b277a186b28a0ac` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:53:17.807405 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-994d1f22-4f11-4a51-b3dc-cd921a5f3df1 None admin] RBAC: Authorizing `identity:get_project(project_id=647d6df216314e948b277a186b28a0ac)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:53:17.808826 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-994d1f22-4f11-4a51-b3dc-cd921a5f3df1 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:53:17.810812 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 549/1103] 10.222.0.40 () {56 vars in 1186 bytes} [Fri Aug 26 10:53:17 2022] GET /identity/v3/projects/647d6df216314e948b277a186b28a0ac => generated 311 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 10:53:17.816522 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994d1f22-4f11-4a51-b3dc-cd921a5f3df1 None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:53:17.883312 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a016932d-9442-44f8-9cbd-057099dfae78 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:53:17.917252 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942a91af-61a6-4f89-bd45-5b55643ccb48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:53:17.917850 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942a91af-61a6-4f89-bd45-5b55643ccb48 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:53:17.918155 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942a91af-61a6-4f89-bd45-5b55643ccb48 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:53:17.918424 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942a91af-61a6-4f89-bd45-5b55643ccb48 None admin] PATH_INFO: `/v3/projects` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:53:17.918735 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994d1f22-4f11-4a51-b3dc-cd921a5f3df1 None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:53:17.919103 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-942a91af-61a6-4f89-bd45-5b55643ccb48 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 10:53:17.919401 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-942a91af-61a6-4f89-bd45-5b55643ccb48 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:53:17.920819 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-942a91af-61a6-4f89-bd45-5b55643ccb48 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:53:17.932608 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:53:17.933693 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 555/1104] 10.222.0.40 () {56 vars in 1148 bytes} [Fri Aug 26 10:53:17 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 10:53:17.939905 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942a91af-61a6-4f89-bd45-5b55643ccb48 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:53:17.953912 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5cd8847-f1a1-4bda-95d0-bcd9e3abab7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:53:17.954950 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5cd8847-f1a1-4bda-95d0-bcd9e3abab7e None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:53:17.955455 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5cd8847-f1a1-4bda-95d0-bcd9e3abab7e None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:53:17.955948 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5cd8847-f1a1-4bda-95d0-bcd9e3abab7e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:53:17.957255 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5cd8847-f1a1-4bda-95d0-bcd9e3abab7e None admin] RBAC: Adding query filter params (role.id=1f97e4602e2e4d6498913a81f9ae8d6a, scope.project.id=647d6df216314e948b277a186b28a0ac, user.id=76e608b7f48c4fe6a152d759ac0f648f) {{(pid=81423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 10:53:17.957818 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5cd8847-f1a1-4bda-95d0-bcd9e3abab7e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:53:17.959638 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5cd8847-f1a1-4bda-95d0-bcd9e3abab7e None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:53:17.966958 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 550/1105] 10.222.0.40 () {56 vars in 1374 bytes} [Fri Aug 26 10:53:17 2022] GET /identity/v3/role_assignments?user.id=76e608b7f48c4fe6a152d759ac0f648f&scope.project.id=647d6df216314e948b277a186b28a0ac&role.id=1f97e4602e2e4d6498913a81f9ae8d6a => generated 608 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 10:53:18.006219 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:53:18.047739 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942a91af-61a6-4f89-bd45-5b55643ccb48 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:53:18.084635 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d7c361-b9b4-4845-b164-9941fd6938f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:53:18.085163 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d7c361-b9b4-4845-b164-9941fd6938f8 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:53:18.085440 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d7c361-b9b4-4845-b164-9941fd6938f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:53:18.085709 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d7c361-b9b4-4845-b164-9941fd6938f8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:53:18.086477 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d7c361-b9b4-4845-b164-9941fd6938f8 None admin] RBAC: Adding query filter params (role.id=3f7d20781437466aabcbf11bda248f3d, scope.project.id=0e30ccf0c1204727b786876c141dbcd2, user.id=1f4b4a662db049ecacc865ecff10bdb0) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 10:53:18.086785 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d7c361-b9b4-4845-b164-9941fd6938f8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:53:18.088226 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d7c361-b9b4-4845-b164-9941fd6938f8 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:53:18.095844 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 556/1106] 10.222.0.40 () {56 vars in 1374 bytes} [Fri Aug 26 10:53:17 2022] GET /identity/v3/role_assignments?user.id=1f4b4a662db049ecacc865ecff10bdb0&scope.project.id=0e30ccf0c1204727b786876c141dbcd2&role.id=3f7d20781437466aabcbf11bda248f3d => generated 608 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 10:53:19.899004 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5c40e2-0d09-4eab-af8c-8d8aca74d17d None None] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:53:19.899433 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5c40e2-0d09-4eab-af8c-8d8aca74d17d None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:53:19.899552 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5c40e2-0d09-4eab-af8c-8d8aca74d17d None None] PATH_INFO: `/` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:53:19.900904 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 551/1107] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 26 10:53:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 10:53:19.907231 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12cad74a-a9ed-4908-9028-0788d13dfd5c None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:53:19.907816 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12cad74a-a9ed-4908-9028-0788d13dfd5c None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:53:19.907816 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12cad74a-a9ed-4908-9028-0788d13dfd5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:53:19.926213 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-12cad74a-a9ed-4908-9028-0788d13dfd5c None None] MFA Rules not processed for user `ca155a48e2024c32af3fd2e19e5fac11`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 10:53:19.927399 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-12cad74a-a9ed-4908-9028-0788d13dfd5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 10:53:19.993412 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 557/1108] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 26 10:53:19 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 26 10:53:20.265775 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5c40e2-0d09-4eab-af8c-8d8aca74d17d None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:53:20.370261 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5c40e2-0d09-4eab-af8c-8d8aca74d17d None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:53:20.402712 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45614b9e-192c-48a4-b59b-c5d9da874159 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:53:20.403323 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45614b9e-192c-48a4-b59b-c5d9da874159 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:53:20.403597 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45614b9e-192c-48a4-b59b-c5d9da874159 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:53:20.403865 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45614b9e-192c-48a4-b59b-c5d9da874159 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:53:20.404961 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45614b9e-192c-48a4-b59b-c5d9da874159 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:53:20.406949 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45614b9e-192c-48a4-b59b-c5d9da874159 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:53:20.408502 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 552/1109] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Aug 26 10:53:20 2022] GET /identity/v3/domains/default => generated 199 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 10:53:20.415182 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12cad74a-a9ed-4908-9028-0788d13dfd5c None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:53:20.471290 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:53:20.511422 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12cad74a-a9ed-4908-9028-0788d13dfd5c None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:53:20.545217 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2304ec95-2bb4-4d9e-aa85-f3dab75c405f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:53:20.545753 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2304ec95-2bb4-4d9e-aa85-f3dab75c405f None admin] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:53:20.546061 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2304ec95-2bb4-4d9e-aa85-f3dab75c405f None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:53:20.546315 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2304ec95-2bb4-4d9e-aa85-f3dab75c405f None admin] PATH_INFO: `/v3/users` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:53:20.547194 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2304ec95-2bb4-4d9e-aa85-f3dab75c405f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:53:20.548665 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2304ec95-2bb4-4d9e-aa85-f3dab75c405f None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:53:20.568879 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 558/1110] 10.222.0.40 () {60 vars in 1132 bytes} [Fri Aug 26 10:53:20 2022] POST /identity/v3/users => generated 286 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 10:53:22.516754 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00c01d9-0d98-45a2-8543-05233f4b4886 None None] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:53:22.517311 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00c01d9-0d98-45a2-8543-05233f4b4886 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:53:22.517311 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00c01d9-0d98-45a2-8543-05233f4b4886 None None] PATH_INFO: `/` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:53:22.518676 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 553/1111] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 26 10:53:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 10:53:22.523838 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8c58e0-a165-4911-a538-5f97d4f5fafd None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:53:22.524255 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8c58e0-a165-4911-a538-5f97d4f5fafd None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:53:22.524371 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8c58e0-a165-4911-a538-5f97d4f5fafd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:53:22.542785 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-5c8c58e0-a165-4911-a538-5f97d4f5fafd None None] MFA Rules not processed for user `ca155a48e2024c32af3fd2e19e5fac11`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 10:53:22.543973 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-5c8c58e0-a165-4911-a538-5f97d4f5fafd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 10:53:22.609587 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 559/1112] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 26 10:53:22 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 26 10:53:22.883115 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00c01d9-0d98-45a2-8543-05233f4b4886 None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:53:22.931737 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:53:22.985165 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00c01d9-0d98-45a2-8543-05233f4b4886 None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:53:23.018348 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ffc455-24a8-400f-b16d-c1ff329095a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:53:23.018954 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ffc455-24a8-400f-b16d-c1ff329095a3 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:53:23.019227 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ffc455-24a8-400f-b16d-c1ff329095a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:53:23.019496 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ffc455-24a8-400f-b16d-c1ff329095a3 None admin] PATH_INFO: `/v3/roles/9c439f898126444f92abb01910f1d02b` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:53:23.020872 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ffc455-24a8-400f-b16d-c1ff329095a3 None admin] RBAC: Authorizing `identity:get_role(role_id=9c439f898126444f92abb01910f1d02b)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:53:23.022241 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ffc455-24a8-400f-b16d-c1ff329095a3 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:53:23.023539 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 554/1113] 10.222.0.40 () {56 vars in 1177 bytes} [Fri Aug 26 10:53:22 2022] GET /identity/v3/roles/9c439f898126444f92abb01910f1d02b => generated 223 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 10:53:23.029255 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8c58e0-a165-4911-a538-5f97d4f5fafd None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:53:23.073912 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:53:23.128444 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8c58e0-a165-4911-a538-5f97d4f5fafd None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:53:23.160543 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf864af0-fc90-4fea-b407-4abb8735f5a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:53:23.161154 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf864af0-fc90-4fea-b407-4abb8735f5a5 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:53:23.161428 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf864af0-fc90-4fea-b407-4abb8735f5a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:53:23.161697 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf864af0-fc90-4fea-b407-4abb8735f5a5 None admin] PATH_INFO: `/v3/users/8f03d92ccf31420dbd8228b73ebdb063` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:53:23.174783 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf864af0-fc90-4fea-b407-4abb8735f5a5 None admin] RBAC: Authorizing `identity:get_user(user_id=8f03d92ccf31420dbd8228b73ebdb063)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:53:23.176202 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf864af0-fc90-4fea-b407-4abb8735f5a5 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:53:23.177761 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 560/1114] 10.222.0.40 () {56 vars in 1177 bytes} [Fri Aug 26 10:53:23 2022] GET /identity/v3/users/8f03d92ccf31420dbd8228b73ebdb063 => generated 286 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 10:53:23.183486 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ffc455-24a8-400f-b16d-c1ff329095a3 None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:53:23.280509 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ffc455-24a8-400f-b16d-c1ff329095a3 None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:53:23.294201 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:53:23.314311 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e77c8c0-d60f-46a7-9203-f047fa7eed56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:53:23.314907 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e77c8c0-d60f-46a7-9203-f047fa7eed56 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:53:23.315185 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e77c8c0-d60f-46a7-9203-f047fa7eed56 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:53:23.315444 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e77c8c0-d60f-46a7-9203-f047fa7eed56 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:53:23.381106 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e77c8c0-d60f-46a7-9203-f047fa7eed56 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:53:23.382521 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e77c8c0-d60f-46a7-9203-f047fa7eed56 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:53:23.478370 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 555/1115] 10.222.0.40 () {58 vars in 1282 bytes} [Fri Aug 26 10:53:23 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 26 10:53:23.485626 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf864af0-fc90-4fea-b407-4abb8735f5a5 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:53:23.582435 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf864af0-fc90-4fea-b407-4abb8735f5a5 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:53:23.583945 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:53:23.617318 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c206c415-bf00-43a3-a172-0ac5eda8ee7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:53:23.617933 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c206c415-bf00-43a3-a172-0ac5eda8ee7f None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:53:23.618237 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c206c415-bf00-43a3-a172-0ac5eda8ee7f None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:53:23.618507 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c206c415-bf00-43a3-a172-0ac5eda8ee7f None admin] PATH_INFO: `/v3/projects/647d6df216314e948b277a186b28a0ac` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:53:23.619603 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c206c415-bf00-43a3-a172-0ac5eda8ee7f None admin] RBAC: Authorizing `identity:get_project(project_id=647d6df216314e948b277a186b28a0ac)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:53:23.621009 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c206c415-bf00-43a3-a172-0ac5eda8ee7f None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:53:23.622649 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 561/1116] 10.222.0.40 () {56 vars in 1186 bytes} [Fri Aug 26 10:53:23 2022] GET /identity/v3/projects/647d6df216314e948b277a186b28a0ac => generated 311 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 10:53:23.629021 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e77c8c0-d60f-46a7-9203-f047fa7eed56 None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:53:23.644037 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:53:23.726375 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e77c8c0-d60f-46a7-9203-f047fa7eed56 None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:53:23.759091 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b702d1-9079-4660-8dd3-d893f04d4097 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:53:23.759675 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b702d1-9079-4660-8dd3-d893f04d4097 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:53:23.759947 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b702d1-9079-4660-8dd3-d893f04d4097 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:53:23.760213 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b702d1-9079-4660-8dd3-d893f04d4097 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:53:23.760916 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b702d1-9079-4660-8dd3-d893f04d4097 None admin] RBAC: Adding query filter params (role.id=9c439f898126444f92abb01910f1d02b, scope.project.id=647d6df216314e948b277a186b28a0ac, user.id=8f03d92ccf31420dbd8228b73ebdb063) {{(pid=81423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 10:53:23.761214 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b702d1-9079-4660-8dd3-d893f04d4097 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:53:23.762642 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b702d1-9079-4660-8dd3-d893f04d4097 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:53:23.768474 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 556/1117] 10.222.0.40 () {56 vars in 1374 bytes} [Fri Aug 26 10:53:23 2022] GET /identity/v3/role_assignments?user.id=8f03d92ccf31420dbd8228b73ebdb063&scope.project.id=647d6df216314e948b277a186b28a0ac&role.id=9c439f898126444f92abb01910f1d02b => generated 259 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 10:53:25.635791 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7af9941-271a-447d-a65d-fd2def25f0a5 None None] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:53:25.636363 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7af9941-271a-447d-a65d-fd2def25f0a5 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:53:25.636363 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7af9941-271a-447d-a65d-fd2def25f0a5 None None] PATH_INFO: `/` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:53:25.637685 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 562/1118] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 26 10:53:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 10:53:25.642851 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7549cdc-e60b-4995-8761-f9f10657da58 None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:53:25.643234 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7549cdc-e60b-4995-8761-f9f10657da58 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:53:25.643359 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7549cdc-e60b-4995-8761-f9f10657da58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:53:25.661394 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-f7549cdc-e60b-4995-8761-f9f10657da58 None None] MFA Rules not processed for user `ca155a48e2024c32af3fd2e19e5fac11`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 10:53:25.662607 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-f7549cdc-e60b-4995-8761-f9f10657da58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 10:53:25.726900 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 557/1119] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 26 10:53:25 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 26 10:53:25.995019 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7af9941-271a-447d-a65d-fd2def25f0a5 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:53:26.094603 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7af9941-271a-447d-a65d-fd2def25f0a5 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:53:26.127907 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b07782-2979-420c-80d0-00b3b388d207 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:53:26.128524 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b07782-2979-420c-80d0-00b3b388d207 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:53:26.128803 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b07782-2979-420c-80d0-00b3b388d207 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:53:26.129546 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b07782-2979-420c-80d0-00b3b388d207 None admin] PATH_INFO: `/v3/roles/9c439f898126444f92abb01910f1d02b` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:53:26.131149 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b07782-2979-420c-80d0-00b3b388d207 None admin] RBAC: Authorizing `identity:get_role(role_id=9c439f898126444f92abb01910f1d02b)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:53:26.132519 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b07782-2979-420c-80d0-00b3b388d207 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:53:26.133815 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 563/1120] 10.222.0.40 () {56 vars in 1177 bytes} [Fri Aug 26 10:53:25 2022] GET /identity/v3/roles/9c439f898126444f92abb01910f1d02b => generated 223 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 10:53:26.139438 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7549cdc-e60b-4995-8761-f9f10657da58 None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:53:26.149391 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:53:26.236937 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7549cdc-e60b-4995-8761-f9f10657da58 None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:53:26.270773 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc454709-1062-4369-8f72-3c5cc12c1143 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:53:26.271324 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc454709-1062-4369-8f72-3c5cc12c1143 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:53:26.271597 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc454709-1062-4369-8f72-3c5cc12c1143 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:53:26.271861 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc454709-1062-4369-8f72-3c5cc12c1143 None admin] PATH_INFO: `/v3/users/8f03d92ccf31420dbd8228b73ebdb063` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:53:26.272956 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc454709-1062-4369-8f72-3c5cc12c1143 None admin] RBAC: Authorizing `identity:get_user(user_id=8f03d92ccf31420dbd8228b73ebdb063)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:53:26.274393 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc454709-1062-4369-8f72-3c5cc12c1143 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:53:26.275945 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 558/1121] 10.222.0.40 () {56 vars in 1177 bytes} [Fri Aug 26 10:53:26 2022] GET /identity/v3/users/8f03d92ccf31420dbd8228b73ebdb063 => generated 286 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 10:53:26.282685 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b07782-2979-420c-80d0-00b3b388d207 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:53:26.321604 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:53:26.380355 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b07782-2979-420c-80d0-00b3b388d207 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:53:26.413756 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0abcf616-64ad-420f-a910-fafcdd812ff1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:53:26.414404 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abcf616-64ad-420f-a910-fafcdd812ff1 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:53:26.414681 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abcf616-64ad-420f-a910-fafcdd812ff1 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:53:26.414954 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abcf616-64ad-420f-a910-fafcdd812ff1 None admin] PATH_INFO: `/v3/projects/647d6df216314e948b277a186b28a0ac` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:53:26.416055 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0abcf616-64ad-420f-a910-fafcdd812ff1 None admin] RBAC: Authorizing `identity:get_project(project_id=647d6df216314e948b277a186b28a0ac)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:53:26.417476 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0abcf616-64ad-420f-a910-fafcdd812ff1 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:53:26.419114 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 564/1122] 10.222.0.40 () {56 vars in 1186 bytes} [Fri Aug 26 10:53:26 2022] GET /identity/v3/projects/647d6df216314e948b277a186b28a0ac => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 10:53:26.425817 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc454709-1062-4369-8f72-3c5cc12c1143 None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:53:26.500979 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:53:26.524294 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc454709-1062-4369-8f72-3c5cc12c1143 None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:53:26.556903 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6519ca07-933f-4033-8a2e-be8725ff9b33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:53:26.557540 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6519ca07-933f-4033-8a2e-be8725ff9b33 None admin] REQUEST_METHOD: `PUT` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:53:26.557817 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6519ca07-933f-4033-8a2e-be8725ff9b33 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:53:26.558116 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6519ca07-933f-4033-8a2e-be8725ff9b33 None admin] PATH_INFO: `/v3/projects/647d6df216314e948b277a186b28a0ac/users/8f03d92ccf31420dbd8228b73ebdb063/roles/9c439f898126444f92abb01910f1d02b` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:53:26.560235 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6519ca07-933f-4033-8a2e-be8725ff9b33 None admin] RBAC: Authorizing `identity:create_grant(project_id=647d6df216314e948b277a186b28a0ac, user_id=8f03d92ccf31420dbd8228b73ebdb063, role_id=9c439f898126444f92abb01910f1d02b)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:53:26.561646 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6519ca07-933f-4033-8a2e-be8725ff9b33 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:53:26.574523 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 559/1123] 10.222.0.40 () {58 vars in 1439 bytes} [Fri Aug 26 10:53:26 2022] PUT /identity/v3/projects/647d6df216314e948b277a186b28a0ac/users/8f03d92ccf31420dbd8228b73ebdb063/roles/9c439f898126444f92abb01910f1d02b => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 10:53:28.441618 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7502df-badc-4b3a-8fb6-d42a2e8836ff None None] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:53:28.442182 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7502df-badc-4b3a-8fb6-d42a2e8836ff None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:53:28.442182 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7502df-badc-4b3a-8fb6-d42a2e8836ff None None] PATH_INFO: `/` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:53:28.443506 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 565/1124] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 26 10:53:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 10:53:28.448709 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30291ab4-63e1-451e-b7c0-16aef85213aa None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:53:28.449089 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30291ab4-63e1-451e-b7c0-16aef85213aa None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:53:28.449263 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30291ab4-63e1-451e-b7c0-16aef85213aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:53:28.467070 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-30291ab4-63e1-451e-b7c0-16aef85213aa None None] MFA Rules not processed for user `ca155a48e2024c32af3fd2e19e5fac11`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 10:53:28.468247 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-30291ab4-63e1-451e-b7c0-16aef85213aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 10:53:28.533862 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 560/1125] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 26 10:53:28 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 26 10:53:28.802426 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7502df-badc-4b3a-8fb6-d42a2e8836ff None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:53:28.901472 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7502df-badc-4b3a-8fb6-d42a2e8836ff None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:53:28.909141 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:53:28.935212 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aebad5fc-9097-4171-810d-ea828c78938c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:53:28.935824 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebad5fc-9097-4171-810d-ea828c78938c None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:53:28.936097 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebad5fc-9097-4171-810d-ea828c78938c None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:53:28.936362 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebad5fc-9097-4171-810d-ea828c78938c None admin] PATH_INFO: `/v3/roles/9c439f898126444f92abb01910f1d02b` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:53:28.937765 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aebad5fc-9097-4171-810d-ea828c78938c None admin] RBAC: Authorizing `identity:get_role(role_id=9c439f898126444f92abb01910f1d02b)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:53:28.939140 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aebad5fc-9097-4171-810d-ea828c78938c None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:53:28.940595 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 566/1126] 10.222.0.40 () {56 vars in 1177 bytes} [Fri Aug 26 10:53:28 2022] GET /identity/v3/roles/9c439f898126444f92abb01910f1d02b => generated 223 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 10:53:28.946246 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30291ab4-63e1-451e-b7c0-16aef85213aa None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:53:29.011528 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:53:29.048805 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30291ab4-63e1-451e-b7c0-16aef85213aa None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:53:29.081594 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db47fc81-d31e-40e1-a5cd-6148022e768f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:53:29.082181 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db47fc81-d31e-40e1-a5cd-6148022e768f None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:53:29.082460 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db47fc81-d31e-40e1-a5cd-6148022e768f None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:53:29.082729 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db47fc81-d31e-40e1-a5cd-6148022e768f None admin] PATH_INFO: `/v3/users/8f03d92ccf31420dbd8228b73ebdb063` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:53:29.083840 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db47fc81-d31e-40e1-a5cd-6148022e768f None admin] RBAC: Authorizing `identity:get_user(user_id=8f03d92ccf31420dbd8228b73ebdb063)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:53:29.085254 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db47fc81-d31e-40e1-a5cd-6148022e768f None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:53:29.086849 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 561/1127] 10.222.0.40 () {56 vars in 1177 bytes} [Fri Aug 26 10:53:28 2022] GET /identity/v3/users/8f03d92ccf31420dbd8228b73ebdb063 => generated 286 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 10:53:29.092756 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aebad5fc-9097-4171-810d-ea828c78938c None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:53:29.191717 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aebad5fc-9097-4171-810d-ea828c78938c None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:53:29.223960 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddbda7be-9178-4dca-b242-c0857a822657 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:53:29.224368 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbda7be-9178-4dca-b242-c0857a822657 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:53:29.224663 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbda7be-9178-4dca-b242-c0857a822657 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:53:29.225015 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbda7be-9178-4dca-b242-c0857a822657 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:53:29.268002 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:53:29.292558 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddbda7be-9178-4dca-b242-c0857a822657 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:53:29.294015 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddbda7be-9178-4dca-b242-c0857a822657 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:53:29.391961 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 567/1128] 10.222.0.40 () {58 vars in 1282 bytes} [Fri Aug 26 10:53:29 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 26 10:53:29.397937 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db47fc81-d31e-40e1-a5cd-6148022e768f None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:53:29.495951 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db47fc81-d31e-40e1-a5cd-6148022e768f None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:53:29.529345 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a535fc0d-8ab9-4b74-a099-dee0138bc401 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:53:29.530213 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a535fc0d-8ab9-4b74-a099-dee0138bc401 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:53:29.530494 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a535fc0d-8ab9-4b74-a099-dee0138bc401 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:53:29.530767 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a535fc0d-8ab9-4b74-a099-dee0138bc401 None admin] PATH_INFO: `/v3/projects/647d6df216314e948b277a186b28a0ac` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:53:29.532420 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a535fc0d-8ab9-4b74-a099-dee0138bc401 None admin] RBAC: Authorizing `identity:get_project(project_id=647d6df216314e948b277a186b28a0ac)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:53:29.533824 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a535fc0d-8ab9-4b74-a099-dee0138bc401 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:53:29.535474 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 562/1129] 10.222.0.40 () {56 vars in 1186 bytes} [Fri Aug 26 10:53:29 2022] GET /identity/v3/projects/647d6df216314e948b277a186b28a0ac => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 10:53:29.541612 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddbda7be-9178-4dca-b242-c0857a822657 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:53:29.613520 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:53:29.637318 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddbda7be-9178-4dca-b242-c0857a822657 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:53:29.669828 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b324f04-35a1-4136-80be-291c47ee9c6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:53:29.670378 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b324f04-35a1-4136-80be-291c47ee9c6a None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:53:29.670651 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b324f04-35a1-4136-80be-291c47ee9c6a None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:53:29.670917 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b324f04-35a1-4136-80be-291c47ee9c6a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:53:29.672296 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b324f04-35a1-4136-80be-291c47ee9c6a None admin] RBAC: Adding query filter params (role.id=9c439f898126444f92abb01910f1d02b, scope.project.id=647d6df216314e948b277a186b28a0ac, user.id=8f03d92ccf31420dbd8228b73ebdb063) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 10:53:29.672601 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b324f04-35a1-4136-80be-291c47ee9c6a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:53:29.674038 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b324f04-35a1-4136-80be-291c47ee9c6a None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:53:29.679532 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 568/1130] 10.222.0.40 () {56 vars in 1374 bytes} [Fri Aug 26 10:53:29 2022] GET /identity/v3/role_assignments?user.id=8f03d92ccf31420dbd8228b73ebdb063&scope.project.id=647d6df216314e948b277a186b28a0ac&role.id=9c439f898126444f92abb01910f1d02b => generated 608 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 10:53:31.593765 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fddc7921-59ee-4f5e-bb73-55762eeed2a1 None None] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:53:31.594325 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fddc7921-59ee-4f5e-bb73-55762eeed2a1 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:53:31.594325 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fddc7921-59ee-4f5e-bb73-55762eeed2a1 None None] PATH_INFO: `/` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:53:31.595665 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 563/1131] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 26 10:53:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 10:53:31.600747 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e415da-75a8-40f1-bba4-10da9b7ef186 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:53:31.601141 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e415da-75a8-40f1-bba4-10da9b7ef186 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:53:31.601286 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e415da-75a8-40f1-bba4-10da9b7ef186 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:53:31.619300 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-39e415da-75a8-40f1-bba4-10da9b7ef186 None None] MFA Rules not processed for user `ca155a48e2024c32af3fd2e19e5fac11`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 10:53:31.620483 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-39e415da-75a8-40f1-bba4-10da9b7ef186 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 10:53:31.684298 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 569/1132] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 26 10:53:31 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 26 10:53:31.951413 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fddc7921-59ee-4f5e-bb73-55762eeed2a1 None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:53:31.954882 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:53:32.059374 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fddc7921-59ee-4f5e-bb73-55762eeed2a1 None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:53:32.092717 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0521aa7-fbd5-4034-bb2e-1ec097320b4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:53:32.093413 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0521aa7-fbd5-4034-bb2e-1ec097320b4c None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:53:32.094242 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0521aa7-fbd5-4034-bb2e-1ec097320b4c None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:53:32.094602 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0521aa7-fbd5-4034-bb2e-1ec097320b4c None admin] PATH_INFO: `/v3/domains/default` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:53:32.095720 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0521aa7-fbd5-4034-bb2e-1ec097320b4c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:53:32.097142 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0521aa7-fbd5-4034-bb2e-1ec097320b4c None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:53:32.098749 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 564/1133] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Aug 26 10:53:31 2022] GET /identity/v3/domains/default => generated 199 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 10:53:32.104541 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e415da-75a8-40f1-bba4-10da9b7ef186 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:53:32.167777 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:53:32.199096 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e415da-75a8-40f1-bba4-10da9b7ef186 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:53:32.232954 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d4cde2-2675-4228-abc7-aee4e8445274 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:53:32.233619 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d4cde2-2675-4228-abc7-aee4e8445274 None admin] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:53:32.233894 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d4cde2-2675-4228-abc7-aee4e8445274 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:53:32.234458 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d4cde2-2675-4228-abc7-aee4e8445274 None admin] PATH_INFO: `/v3/projects` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:53:32.235196 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d4cde2-2675-4228-abc7-aee4e8445274 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:53:32.236598 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d4cde2-2675-4228-abc7-aee4e8445274 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:53:32.261772 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 570/1134] 10.222.0.40 () {60 vars in 1141 bytes} [Fri Aug 26 10:53:32 2022] POST /identity/v3/projects => generated 311 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 10:53:34.201463 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8061a622-0278-48ae-a6bf-3224fac1b69a None None] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:53:34.201959 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8061a622-0278-48ae-a6bf-3224fac1b69a None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:53:34.201959 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8061a622-0278-48ae-a6bf-3224fac1b69a None None] PATH_INFO: `/` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:53:34.203805 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 565/1135] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 26 10:53:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 10:53:34.208953 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87304e85-3ef9-42a5-93dd-024486a00290 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:53:34.209379 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87304e85-3ef9-42a5-93dd-024486a00290 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:53:34.209556 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87304e85-3ef9-42a5-93dd-024486a00290 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:53:34.227944 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-87304e85-3ef9-42a5-93dd-024486a00290 None None] MFA Rules not processed for user `ca155a48e2024c32af3fd2e19e5fac11`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 10:53:34.229135 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-87304e85-3ef9-42a5-93dd-024486a00290 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 10:53:34.293762 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 571/1136] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 26 10:53:34 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 26 10:53:34.567540 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8061a622-0278-48ae-a6bf-3224fac1b69a None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:53:34.637347 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:53:34.668909 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8061a622-0278-48ae-a6bf-3224fac1b69a None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:53:34.701930 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e098c099-96a8-49e5-823a-d9d560cf778b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:53:34.702570 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e098c099-96a8-49e5-823a-d9d560cf778b None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:53:34.703391 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e098c099-96a8-49e5-823a-d9d560cf778b None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:53:34.703663 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e098c099-96a8-49e5-823a-d9d560cf778b None admin] PATH_INFO: `/v3/domains/default` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:53:34.704851 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e098c099-96a8-49e5-823a-d9d560cf778b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:53:34.706282 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e098c099-96a8-49e5-823a-d9d560cf778b None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:53:34.707832 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 566/1137] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Aug 26 10:53:34 2022] GET /identity/v3/domains/default => generated 199 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 10:53:34.713716 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87304e85-3ef9-42a5-93dd-024486a00290 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:53:34.762053 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:53:34.811157 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87304e85-3ef9-42a5-93dd-024486a00290 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:53:34.844687 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01941a52-9fe3-4a12-b529-c5a4597af750 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:53:34.845224 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01941a52-9fe3-4a12-b529-c5a4597af750 None admin] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:53:34.845501 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01941a52-9fe3-4a12-b529-c5a4597af750 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:53:34.845769 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01941a52-9fe3-4a12-b529-c5a4597af750 None admin] PATH_INFO: `/v3/users` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:53:34.846440 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01941a52-9fe3-4a12-b529-c5a4597af750 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:53:34.847861 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01941a52-9fe3-4a12-b529-c5a4597af750 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:53:34.868373 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 572/1138] 10.222.0.40 () {60 vars in 1132 bytes} [Fri Aug 26 10:53:34 2022] POST /identity/v3/users => generated 286 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 26 10:53:36.810447 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5ad5b2-6978-48f8-b007-978cc9b08897 None None] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:53:36.811018 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5ad5b2-6978-48f8-b007-978cc9b08897 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:53:36.811018 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5ad5b2-6978-48f8-b007-978cc9b08897 None None] PATH_INFO: `/` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:53:36.812400 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 567/1139] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 26 10:53:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 10:53:36.817651 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c4f341-c617-418e-b12e-2cb15c7f192f None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:53:36.818042 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c4f341-c617-418e-b12e-2cb15c7f192f None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:53:36.818229 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c4f341-c617-418e-b12e-2cb15c7f192f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:53:36.836354 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-f9c4f341-c617-418e-b12e-2cb15c7f192f None None] MFA Rules not processed for user `ca155a48e2024c32af3fd2e19e5fac11`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 10:53:36.837535 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-f9c4f341-c617-418e-b12e-2cb15c7f192f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 10:53:36.902629 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 573/1140] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 26 10:53:36 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 26 10:53:37.171000 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5ad5b2-6978-48f8-b007-978cc9b08897 None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:53:37.271131 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5ad5b2-6978-48f8-b007-978cc9b08897 None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:53:37.303811 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84056d6-6156-474d-9af6-d702a9861038 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:53:37.304422 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84056d6-6156-474d-9af6-d702a9861038 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:53:37.304699 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84056d6-6156-474d-9af6-d702a9861038 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:53:37.304968 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84056d6-6156-474d-9af6-d702a9861038 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:53:37.311551 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:53:37.315632 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c84056d6-6156-474d-9af6-d702a9861038 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 10:53:37.320465 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application [None req-c84056d6-6156-474d-9af6-d702a9861038 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 26 10:53:37.320465 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 10:53:37.320465 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 10:53:37.320465 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 10:53:37.320465 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 10:53:37.320465 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 10:53:37.320465 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 10:53:37.320465 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 10:53:37.320465 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 10:53:37.320465 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 10:53:37.320465 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 10:53:37.320465 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 10:53:37.320465 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 10:53:37.320465 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 10:53:37.320465 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 10:53:37.320465 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 10:53:37.321648 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 10:53:37.321648 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 10:53:37.321648 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 10:53:37.321648 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 10:53:37.321648 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 10:53:37.321648 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 10:53:37.321648 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 10:53:37.321648 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 10:53:37.321648 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 10:53:37.321648 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 10:53:37.321648 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 10:53:37.321648 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application with Lock( Aug 26 10:53:37.321648 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 10:53:37.321648 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._enter() Aug 26 10:53:37.321648 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 10:53:37.321648 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 10:53:37.322799 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 10:53:37.322799 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.creator() Aug 26 10:53:37.322799 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 10:53:37.322799 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application created_value = creator( Aug 26 10:53:37.322799 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 10:53:37.322799 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 10:53:37.322799 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 10:53:37.322799 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 10:53:37.322799 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 10:53:37.322799 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 10:53:37.322799 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 26 10:53:37.322799 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Aug 26 10:53:37.322799 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 568/1141] 10.222.0.40 () {56 vars in 1096 bytes} [Fri Aug 26 10:53:37 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 10:53:37.328202 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c4f341-c617-418e-b12e-2cb15c7f192f None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:53:37.361707 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:53:37.424514 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c4f341-c617-418e-b12e-2cb15c7f192f None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:53:37.456972 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4920704-4eca-4e2c-b69c-f2fc9bcef9bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:53:37.457566 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4920704-4eca-4e2c-b69c-f2fc9bcef9bc None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:53:37.457843 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4920704-4eca-4e2c-b69c-f2fc9bcef9bc None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:53:37.458141 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4920704-4eca-4e2c-b69c-f2fc9bcef9bc None admin] PATH_INFO: `/v3/roles` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:53:37.458792 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4920704-4eca-4e2c-b69c-f2fc9bcef9bc None admin] RBAC: Adding query filter params (name=admin) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 10:53:37.459099 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4920704-4eca-4e2c-b69c-f2fc9bcef9bc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:53:37.460437 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4920704-4eca-4e2c-b69c-f2fc9bcef9bc None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:53:37.468210 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 574/1142] 10.222.0.40 () {56 vars in 1099 bytes} [Fri Aug 26 10:53:37 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 10:53:37.474670 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84056d6-6156-474d-9af6-d702a9861038 None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:53:37.571862 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84056d6-6156-474d-9af6-d702a9861038 None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:53:37.604830 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40cbd2c7-890f-4276-b662-01c4da0f4ccb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:53:37.605438 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40cbd2c7-890f-4276-b662-01c4da0f4ccb None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:53:37.605715 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40cbd2c7-890f-4276-b662-01c4da0f4ccb None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:53:37.606000 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40cbd2c7-890f-4276-b662-01c4da0f4ccb None admin] PATH_INFO: `/v3/users/7aff95667a67448fb0c1fd57c78a5b8c` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:53:37.618763 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40cbd2c7-890f-4276-b662-01c4da0f4ccb None admin] RBAC: Authorizing `identity:get_user(user_id=7aff95667a67448fb0c1fd57c78a5b8c)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:53:37.620190 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40cbd2c7-890f-4276-b662-01c4da0f4ccb None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:53:37.621743 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 569/1143] 10.222.0.40 () {56 vars in 1177 bytes} [Fri Aug 26 10:53:37 2022] GET /identity/v3/users/7aff95667a67448fb0c1fd57c78a5b8c => generated 286 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 10:53:37.628250 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4920704-4eca-4e2c-b69c-f2fc9bcef9bc None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:53:37.722543 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:53:37.726522 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4920704-4eca-4e2c-b69c-f2fc9bcef9bc None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:53:37.762337 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec083d1-e8f3-4621-9705-2665888bd7f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:53:37.762934 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec083d1-e8f3-4621-9705-2665888bd7f1 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:53:37.763211 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec083d1-e8f3-4621-9705-2665888bd7f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:53:37.763484 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec083d1-e8f3-4621-9705-2665888bd7f1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:53:37.828971 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec083d1-e8f3-4621-9705-2665888bd7f1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:53:37.830445 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec083d1-e8f3-4621-9705-2665888bd7f1 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:53:37.922790 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:53:37.927695 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 575/1144] 10.222.0.40 () {58 vars in 1282 bytes} [Fri Aug 26 10:53:37 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 26 10:53:37.933398 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40cbd2c7-890f-4276-b662-01c4da0f4ccb None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:53:37.988391 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:53:38.035618 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40cbd2c7-890f-4276-b662-01c4da0f4ccb None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:53:38.070576 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c1bfdc-4285-41a1-9708-72a445575fe3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:53:38.071193 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c1bfdc-4285-41a1-9708-72a445575fe3 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:53:38.071472 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c1bfdc-4285-41a1-9708-72a445575fe3 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:53:38.071744 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c1bfdc-4285-41a1-9708-72a445575fe3 None admin] PATH_INFO: `/v3/projects/c53673106b8f4e46a28b7dc57da80075` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:53:38.072857 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c1bfdc-4285-41a1-9708-72a445575fe3 None admin] RBAC: Authorizing `identity:get_project(project_id=c53673106b8f4e46a28b7dc57da80075)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:53:38.074300 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c1bfdc-4285-41a1-9708-72a445575fe3 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:53:38.075927 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 570/1145] 10.222.0.40 () {56 vars in 1186 bytes} [Fri Aug 26 10:53:37 2022] GET /identity/v3/projects/c53673106b8f4e46a28b7dc57da80075 => generated 311 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 10:53:38.082148 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec083d1-e8f3-4621-9705-2665888bd7f1 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:53:38.177118 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec083d1-e8f3-4621-9705-2665888bd7f1 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:53:38.209690 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546b5002-b62c-49ea-8447-c5a0bdb430a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:53:38.210303 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546b5002-b62c-49ea-8447-c5a0bdb430a9 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:53:38.210579 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546b5002-b62c-49ea-8447-c5a0bdb430a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:53:38.210847 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546b5002-b62c-49ea-8447-c5a0bdb430a9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:53:38.211555 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-546b5002-b62c-49ea-8447-c5a0bdb430a9 None admin] RBAC: Adding query filter params (role.id=1f97e4602e2e4d6498913a81f9ae8d6a, scope.project.id=c53673106b8f4e46a28b7dc57da80075, user.id=7aff95667a67448fb0c1fd57c78a5b8c) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 10:53:38.211858 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-546b5002-b62c-49ea-8447-c5a0bdb430a9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:53:38.213266 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-546b5002-b62c-49ea-8447-c5a0bdb430a9 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:53:38.218488 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 576/1146] 10.222.0.40 () {56 vars in 1374 bytes} [Fri Aug 26 10:53:38 2022] GET /identity/v3/role_assignments?user.id=7aff95667a67448fb0c1fd57c78a5b8c&scope.project.id=c53673106b8f4e46a28b7dc57da80075&role.id=1f97e4602e2e4d6498913a81f9ae8d6a => generated 259 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 10:53:40.101624 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0119b78-9657-4ef6-b6eb-1dc285377a57 None None] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:53:40.102171 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0119b78-9657-4ef6-b6eb-1dc285377a57 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:53:40.102292 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0119b78-9657-4ef6-b6eb-1dc285377a57 None None] PATH_INFO: `/` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:53:40.103548 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 571/1147] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 26 10:53:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 10:53:40.109502 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb38d1b7-d6f1-4c57-a0f3-76eca47779fe None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:53:40.109966 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb38d1b7-d6f1-4c57-a0f3-76eca47779fe None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:53:40.110138 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb38d1b7-d6f1-4c57-a0f3-76eca47779fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:53:40.127498 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-eb38d1b7-d6f1-4c57-a0f3-76eca47779fe None None] MFA Rules not processed for user `ca155a48e2024c32af3fd2e19e5fac11`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 10:53:40.128604 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-eb38d1b7-d6f1-4c57-a0f3-76eca47779fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 10:53:40.182825 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:53:40.192934 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 577/1148] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 26 10:53:40 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 26 10:53:40.462333 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0119b78-9657-4ef6-b6eb-1dc285377a57 None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:53:40.562589 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0119b78-9657-4ef6-b6eb-1dc285377a57 None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:53:40.585657 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:53:40.596765 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73742207-8f75-4631-845b-83f6a853737c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:53:40.597380 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73742207-8f75-4631-845b-83f6a853737c None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:53:40.597657 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73742207-8f75-4631-845b-83f6a853737c None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:53:40.597928 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73742207-8f75-4631-845b-83f6a853737c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:53:40.607829 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73742207-8f75-4631-845b-83f6a853737c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 10:53:40.612808 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application [None req-73742207-8f75-4631-845b-83f6a853737c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 26 10:53:40.612808 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 10:53:40.612808 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 10:53:40.612808 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 10:53:40.612808 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 10:53:40.612808 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 10:53:40.612808 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 10:53:40.612808 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 10:53:40.612808 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 10:53:40.612808 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 10:53:40.612808 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 10:53:40.612808 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 10:53:40.612808 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 10:53:40.612808 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 10:53:40.612808 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 10:53:40.612808 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 10:53:40.613886 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 10:53:40.613886 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 10:53:40.613886 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 10:53:40.613886 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 10:53:40.613886 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 10:53:40.613886 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 10:53:40.613886 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 10:53:40.613886 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 10:53:40.613886 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 10:53:40.613886 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 10:53:40.613886 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 10:53:40.613886 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application with Lock( Aug 26 10:53:40.613886 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 10:53:40.613886 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._enter() Aug 26 10:53:40.613886 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 10:53:40.613886 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 10:53:40.614952 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 10:53:40.614952 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.creator() Aug 26 10:53:40.614952 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 10:53:40.614952 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application created_value = creator( Aug 26 10:53:40.614952 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 10:53:40.614952 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 10:53:40.614952 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 10:53:40.614952 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 10:53:40.614952 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 10:53:40.614952 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 10:53:40.614952 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 26 10:53:40.614952 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Aug 26 10:53:40.614952 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 572/1149] 10.222.0.40 () {56 vars in 1096 bytes} [Fri Aug 26 10:53:40 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 10:53:40.619923 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb38d1b7-d6f1-4c57-a0f3-76eca47779fe None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:53:40.714979 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb38d1b7-d6f1-4c57-a0f3-76eca47779fe None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:53:40.746854 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bceed898-6145-4f27-a209-660bc4ad8ff1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:53:40.747440 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bceed898-6145-4f27-a209-660bc4ad8ff1 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:53:40.747713 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bceed898-6145-4f27-a209-660bc4ad8ff1 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:53:40.747979 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bceed898-6145-4f27-a209-660bc4ad8ff1 None admin] PATH_INFO: `/v3/roles` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:53:40.748626 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bceed898-6145-4f27-a209-660bc4ad8ff1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 10:53:40.748921 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bceed898-6145-4f27-a209-660bc4ad8ff1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:53:40.750798 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bceed898-6145-4f27-a209-660bc4ad8ff1 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:53:40.758411 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 578/1150] 10.222.0.40 () {56 vars in 1099 bytes} [Fri Aug 26 10:53:40 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 10:53:40.764207 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73742207-8f75-4631-845b-83f6a853737c None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:53:40.863429 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73742207-8f75-4631-845b-83f6a853737c None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:53:40.897731 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a845ec1-1bb9-43c9-9183-9c6f2d121256 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:53:40.898376 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a845ec1-1bb9-43c9-9183-9c6f2d121256 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:53:40.898650 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a845ec1-1bb9-43c9-9183-9c6f2d121256 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:53:40.898919 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a845ec1-1bb9-43c9-9183-9c6f2d121256 None admin] PATH_INFO: `/v3/users/7aff95667a67448fb0c1fd57c78a5b8c` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:53:40.900023 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a845ec1-1bb9-43c9-9183-9c6f2d121256 None admin] RBAC: Authorizing `identity:get_user(user_id=7aff95667a67448fb0c1fd57c78a5b8c)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:53:40.901439 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a845ec1-1bb9-43c9-9183-9c6f2d121256 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:53:40.903043 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 573/1151] 10.222.0.40 () {56 vars in 1177 bytes} [Fri Aug 26 10:53:40 2022] GET /identity/v3/users/7aff95667a67448fb0c1fd57c78a5b8c => generated 286 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 10:53:40.909299 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bceed898-6145-4f27-a209-660bc4ad8ff1 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:53:40.964386 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:53:41.009373 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bceed898-6145-4f27-a209-660bc4ad8ff1 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:53:41.041705 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dedfd502-aec7-4b75-9257-d9b9794ee737 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:53:41.042356 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dedfd502-aec7-4b75-9257-d9b9794ee737 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:53:41.042636 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dedfd502-aec7-4b75-9257-d9b9794ee737 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:53:41.042910 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dedfd502-aec7-4b75-9257-d9b9794ee737 None admin] PATH_INFO: `/v3/projects/c53673106b8f4e46a28b7dc57da80075` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:53:41.044011 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dedfd502-aec7-4b75-9257-d9b9794ee737 None admin] RBAC: Authorizing `identity:get_project(project_id=c53673106b8f4e46a28b7dc57da80075)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:53:41.045433 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dedfd502-aec7-4b75-9257-d9b9794ee737 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:53:41.048118 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 579/1152] 10.222.0.40 () {56 vars in 1186 bytes} [Fri Aug 26 10:53:40 2022] GET /identity/v3/projects/c53673106b8f4e46a28b7dc57da80075 => generated 311 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 10:53:41.054395 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a845ec1-1bb9-43c9-9183-9c6f2d121256 None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:53:41.105582 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:53:41.152164 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a845ec1-1bb9-43c9-9183-9c6f2d121256 None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:53:41.184553 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61eadcea-bf10-4feb-85e7-6e5e498af212 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:53:41.185194 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61eadcea-bf10-4feb-85e7-6e5e498af212 None admin] REQUEST_METHOD: `PUT` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:53:41.185467 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61eadcea-bf10-4feb-85e7-6e5e498af212 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:53:41.185735 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61eadcea-bf10-4feb-85e7-6e5e498af212 None admin] PATH_INFO: `/v3/projects/c53673106b8f4e46a28b7dc57da80075/users/7aff95667a67448fb0c1fd57c78a5b8c/roles/1f97e4602e2e4d6498913a81f9ae8d6a` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:53:41.188180 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61eadcea-bf10-4feb-85e7-6e5e498af212 None admin] RBAC: Authorizing `identity:create_grant(project_id=c53673106b8f4e46a28b7dc57da80075, user_id=7aff95667a67448fb0c1fd57c78a5b8c, role_id=1f97e4602e2e4d6498913a81f9ae8d6a)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:53:41.189598 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61eadcea-bf10-4feb-85e7-6e5e498af212 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:53:41.202296 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 574/1153] 10.222.0.40 () {58 vars in 1439 bytes} [Fri Aug 26 10:53:41 2022] PUT /identity/v3/projects/c53673106b8f4e46a28b7dc57da80075/users/7aff95667a67448fb0c1fd57c78a5b8c/roles/1f97e4602e2e4d6498913a81f9ae8d6a => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 10:53:43.253742 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678b7b47-84c8-4da6-8da6-4ba5e0ed0b50 None None] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:53:43.254326 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678b7b47-84c8-4da6-8da6-4ba5e0ed0b50 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:53:43.254326 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678b7b47-84c8-4da6-8da6-4ba5e0ed0b50 None None] PATH_INFO: `/` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:53:43.255551 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 580/1154] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 26 10:53:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 10:53:43.260782 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df37f63-f627-4704-92f0-fb7aef1f78f1 None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:53:43.261234 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df37f63-f627-4704-92f0-fb7aef1f78f1 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:53:43.261329 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df37f63-f627-4704-92f0-fb7aef1f78f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:53:43.279021 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-0df37f63-f627-4704-92f0-fb7aef1f78f1 None None] MFA Rules not processed for user `ca155a48e2024c32af3fd2e19e5fac11`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 10:53:43.280193 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-0df37f63-f627-4704-92f0-fb7aef1f78f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 10:53:43.344783 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 575/1155] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 26 10:53:43 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 26 10:53:43.617600 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678b7b47-84c8-4da6-8da6-4ba5e0ed0b50 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:53:43.713709 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678b7b47-84c8-4da6-8da6-4ba5e0ed0b50 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:53:43.736908 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:53:43.748297 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1383b748-e9df-4c19-be35-e1d46c60b42d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:53:43.748905 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1383b748-e9df-4c19-be35-e1d46c60b42d None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:53:43.749178 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1383b748-e9df-4c19-be35-e1d46c60b42d None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:53:43.749487 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1383b748-e9df-4c19-be35-e1d46c60b42d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:53:43.759377 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1383b748-e9df-4c19-be35-e1d46c60b42d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 10:53:43.763985 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application [None req-1383b748-e9df-4c19-be35-e1d46c60b42d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 26 10:53:43.763985 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 10:53:43.763985 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 10:53:43.763985 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 10:53:43.763985 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 10:53:43.763985 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 10:53:43.763985 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 10:53:43.763985 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 10:53:43.763985 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 10:53:43.763985 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 10:53:43.763985 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 10:53:43.763985 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 10:53:43.763985 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 10:53:43.763985 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 10:53:43.763985 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 10:53:43.763985 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 10:53:43.765085 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 10:53:43.765085 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 10:53:43.765085 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 10:53:43.765085 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 10:53:43.765085 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 10:53:43.765085 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 10:53:43.765085 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 10:53:43.765085 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 10:53:43.765085 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 10:53:43.765085 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 10:53:43.765085 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 10:53:43.765085 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application with Lock( Aug 26 10:53:43.765085 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 10:53:43.765085 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._enter() Aug 26 10:53:43.765085 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 10:53:43.765085 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 10:53:43.766472 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 10:53:43.766472 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.creator() Aug 26 10:53:43.766472 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 10:53:43.766472 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application created_value = creator( Aug 26 10:53:43.766472 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 10:53:43.766472 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 10:53:43.766472 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 10:53:43.766472 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 10:53:43.766472 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 10:53:43.766472 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 10:53:43.766472 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 26 10:53:43.766472 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Aug 26 10:53:43.766472 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 581/1156] 10.222.0.40 () {56 vars in 1096 bytes} [Fri Aug 26 10:53:43 2022] GET /identity/v3/roles/admin => generated 83 bytes in 148 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 10:53:43.770991 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df37f63-f627-4704-92f0-fb7aef1f78f1 None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:53:43.813319 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:53:43.869243 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df37f63-f627-4704-92f0-fb7aef1f78f1 None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:53:43.901756 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed8c990-916f-4d0a-b1ea-d07fd3b76ab7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:53:43.902502 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed8c990-916f-4d0a-b1ea-d07fd3b76ab7 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:53:43.902778 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed8c990-916f-4d0a-b1ea-d07fd3b76ab7 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:53:43.903046 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed8c990-916f-4d0a-b1ea-d07fd3b76ab7 None admin] PATH_INFO: `/v3/roles` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:53:43.903707 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed8c990-916f-4d0a-b1ea-d07fd3b76ab7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 10:53:43.904013 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed8c990-916f-4d0a-b1ea-d07fd3b76ab7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:53:43.905357 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed8c990-916f-4d0a-b1ea-d07fd3b76ab7 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:53:43.914702 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 576/1157] 10.222.0.40 () {56 vars in 1099 bytes} [Fri Aug 26 10:53:43 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 10:53:43.920461 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1383b748-e9df-4c19-be35-e1d46c60b42d None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:53:44.017104 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1383b748-e9df-4c19-be35-e1d46c60b42d None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:53:44.051296 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bedf2af-7ebc-4e2e-bf72-78b9b9ba5715 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:53:44.051908 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bedf2af-7ebc-4e2e-bf72-78b9b9ba5715 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:53:44.052182 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bedf2af-7ebc-4e2e-bf72-78b9b9ba5715 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:53:44.052448 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bedf2af-7ebc-4e2e-bf72-78b9b9ba5715 None admin] PATH_INFO: `/v3/users/7aff95667a67448fb0c1fd57c78a5b8c` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:53:44.053545 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bedf2af-7ebc-4e2e-bf72-78b9b9ba5715 None admin] RBAC: Authorizing `identity:get_user(user_id=7aff95667a67448fb0c1fd57c78a5b8c)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:53:44.054984 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bedf2af-7ebc-4e2e-bf72-78b9b9ba5715 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:53:44.056536 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 582/1158] 10.222.0.40 () {56 vars in 1177 bytes} [Fri Aug 26 10:53:43 2022] GET /identity/v3/users/7aff95667a67448fb0c1fd57c78a5b8c => generated 286 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 10:53:44.062492 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed8c990-916f-4d0a-b1ea-d07fd3b76ab7 None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:53:44.160262 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed8c990-916f-4d0a-b1ea-d07fd3b76ab7 None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:53:44.162532 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:53:44.193591 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34c23a9-3e4e-4eb5-aa2f-7d9e4050812b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:53:44.194226 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34c23a9-3e4e-4eb5-aa2f-7d9e4050812b None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:53:44.194504 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34c23a9-3e4e-4eb5-aa2f-7d9e4050812b None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:53:44.194774 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34c23a9-3e4e-4eb5-aa2f-7d9e4050812b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:53:44.260230 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c34c23a9-3e4e-4eb5-aa2f-7d9e4050812b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:53:44.261657 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c34c23a9-3e4e-4eb5-aa2f-7d9e4050812b None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:53:44.360180 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 577/1159] 10.222.0.40 () {58 vars in 1282 bytes} [Fri Aug 26 10:53:44 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 26 10:53:44.366151 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bedf2af-7ebc-4e2e-bf72-78b9b9ba5715 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:53:44.415558 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:53:44.467415 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bedf2af-7ebc-4e2e-bf72-78b9b9ba5715 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:53:44.501475 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e1082d-354f-45ba-9016-40f67f75c2e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:53:44.502116 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e1082d-354f-45ba-9016-40f67f75c2e8 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:53:44.502390 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e1082d-354f-45ba-9016-40f67f75c2e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:53:44.502658 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e1082d-354f-45ba-9016-40f67f75c2e8 None admin] PATH_INFO: `/v3/projects/c53673106b8f4e46a28b7dc57da80075` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:53:44.503761 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e1082d-354f-45ba-9016-40f67f75c2e8 None admin] RBAC: Authorizing `identity:get_project(project_id=c53673106b8f4e46a28b7dc57da80075)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:53:44.505179 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e1082d-354f-45ba-9016-40f67f75c2e8 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:53:44.506804 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 583/1160] 10.222.0.40 () {56 vars in 1186 bytes} [Fri Aug 26 10:53:44 2022] GET /identity/v3/projects/c53673106b8f4e46a28b7dc57da80075 => generated 311 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 10:53:44.512973 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34c23a9-3e4e-4eb5-aa2f-7d9e4050812b None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:53:44.516046 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:53:44.608384 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34c23a9-3e4e-4eb5-aa2f-7d9e4050812b None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:53:44.641316 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408275e8-a434-4d07-adbf-6fc107397445 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:53:44.641902 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408275e8-a434-4d07-adbf-6fc107397445 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:53:44.642208 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408275e8-a434-4d07-adbf-6fc107397445 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:53:44.642481 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408275e8-a434-4d07-adbf-6fc107397445 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:53:44.643194 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-408275e8-a434-4d07-adbf-6fc107397445 None admin] RBAC: Adding query filter params (role.id=1f97e4602e2e4d6498913a81f9ae8d6a, scope.project.id=c53673106b8f4e46a28b7dc57da80075, user.id=7aff95667a67448fb0c1fd57c78a5b8c) {{(pid=81423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 10:53:44.643497 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-408275e8-a434-4d07-adbf-6fc107397445 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:53:44.644903 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-408275e8-a434-4d07-adbf-6fc107397445 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:53:44.650457 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 578/1161] 10.222.0.40 () {56 vars in 1374 bytes} [Fri Aug 26 10:53:44 2022] GET /identity/v3/role_assignments?user.id=7aff95667a67448fb0c1fd57c78a5b8c&scope.project.id=c53673106b8f4e46a28b7dc57da80075&role.id=1f97e4602e2e4d6498913a81f9ae8d6a => generated 608 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 10:53:46.588989 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bda58e-ba46-4525-91ea-f99242c0a19e None None] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:53:46.589512 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bda58e-ba46-4525-91ea-f99242c0a19e None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:53:46.589512 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bda58e-ba46-4525-91ea-f99242c0a19e None None] PATH_INFO: `/` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:53:46.590890 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 584/1162] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 26 10:53:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 10:53:46.596616 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471005c1-1565-4585-8a64-74126f994b97 None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:53:46.597030 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471005c1-1565-4585-8a64-74126f994b97 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:53:46.597162 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471005c1-1565-4585-8a64-74126f994b97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:53:46.615738 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-471005c1-1565-4585-8a64-74126f994b97 None None] MFA Rules not processed for user `ca155a48e2024c32af3fd2e19e5fac11`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 10:53:46.616922 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-471005c1-1565-4585-8a64-74126f994b97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 10:53:46.677327 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:53:46.683148 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 579/1163] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 26 10:53:46 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 26 10:53:46.956179 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01bda58e-ba46-4525-91ea-f99242c0a19e None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:53:47.055702 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01bda58e-ba46-4525-91ea-f99242c0a19e None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:53:47.072151 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:53:47.089822 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1bd885-449f-4a34-a2f0-507de3ceaf47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:53:47.090454 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1bd885-449f-4a34-a2f0-507de3ceaf47 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:53:47.090734 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1bd885-449f-4a34-a2f0-507de3ceaf47 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:53:47.091008 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1bd885-449f-4a34-a2f0-507de3ceaf47 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:53:47.161081 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b1bd885-449f-4a34-a2f0-507de3ceaf47 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:53:47.162482 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b1bd885-449f-4a34-a2f0-507de3ceaf47 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:53:47.256339 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 585/1164] 10.222.0.40 () {58 vars in 1282 bytes} [Fri Aug 26 10:53:46 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 26 10:53:47.262288 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471005c1-1565-4585-8a64-74126f994b97 None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:53:47.357033 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471005c1-1565-4585-8a64-74126f994b97 None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:53:47.389909 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c005c4-0bec-4a97-9ef6-07c41b055a14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:53:47.391114 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c005c4-0bec-4a97-9ef6-07c41b055a14 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:53:47.391385 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c005c4-0bec-4a97-9ef6-07c41b055a14 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:53:47.391653 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c005c4-0bec-4a97-9ef6-07c41b055a14 None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:53:47.398102 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c005c4-0bec-4a97-9ef6-07c41b055a14 None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:53:47.399519 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c005c4-0bec-4a97-9ef6-07c41b055a14 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:53:47.410465 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application [None req-66c005c4-0bec-4a97-9ef6-07c41b055a14 None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Aug 26 10:53:47.410465 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 10:53:47.410465 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Aug 26 10:53:47.410465 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Aug 26 10:53:47.410465 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 26 10:53:47.410465 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 26 10:53:47.410465 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 26 10:53:47.410465 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 26 10:53:47.410465 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Aug 26 10:53:47.410465 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Aug 26 10:53:47.410465 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 26 10:53:47.410465 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Aug 26 10:53:47.410465 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 10:53:47.410465 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 10:53:47.410465 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 10:53:47.410465 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 10:53:47.410465 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 10:53:47.411750 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 10:53:47.411750 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 10:53:47.411750 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 10:53:47.411750 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 10:53:47.411750 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 10:53:47.411750 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 10:53:47.411750 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 10:53:47.411750 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 10:53:47.411750 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 10:53:47.411750 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 10:53:47.411750 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 10:53:47.411750 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 10:53:47.411750 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 10:53:47.411750 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 10:53:47.411750 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 10:53:47.411750 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 10:53:47.412851 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 10:53:47.412851 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application with Lock( Aug 26 10:53:47.412851 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 10:53:47.412851 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._enter() Aug 26 10:53:47.412851 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 10:53:47.412851 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 10:53:47.412851 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 10:53:47.412851 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.creator() Aug 26 10:53:47.412851 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 10:53:47.412851 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application created_value = creator( Aug 26 10:53:47.412851 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Aug 26 10:53:47.412851 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 10:53:47.412851 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Aug 26 10:53:47.412851 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Aug 26 10:53:47.412851 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 580/1165] 10.222.0.40 () {56 vars in 1117 bytes} [Fri Aug 26 10:53:47 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 26 10:53:47.417548 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1bd885-449f-4a34-a2f0-507de3ceaf47 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:53:47.435898 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:53:47.520956 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1bd885-449f-4a34-a2f0-507de3ceaf47 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:53:47.553011 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bffd886-3694-4c00-8f45-2d8a3ba3cac3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:53:47.553600 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bffd886-3694-4c00-8f45-2d8a3ba3cac3 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:53:47.553871 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bffd886-3694-4c00-8f45-2d8a3ba3cac3 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:53:47.554163 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bffd886-3694-4c00-8f45-2d8a3ba3cac3 None admin] PATH_INFO: `/v3/domains` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:53:47.554785 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bffd886-3694-4c00-8f45-2d8a3ba3cac3 None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 10:53:47.555077 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bffd886-3694-4c00-8f45-2d8a3ba3cac3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:53:47.556418 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bffd886-3694-4c00-8f45-2d8a3ba3cac3 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:53:47.564030 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 586/1166] 10.222.0.40 () {56 vars in 1115 bytes} [Fri Aug 26 10:53:47 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 10:53:49.440064 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4112f69f-e8a7-4295-84ac-c657f2b271c5 None None] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:53:49.440606 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4112f69f-e8a7-4295-84ac-c657f2b271c5 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:53:49.440722 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4112f69f-e8a7-4295-84ac-c657f2b271c5 None None] PATH_INFO: `/` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:53:49.442014 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 581/1167] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 26 10:53:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 10:53:49.447366 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95de6bf1-882f-400a-b802-2ade7ad0beca None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:53:49.447774 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95de6bf1-882f-400a-b802-2ade7ad0beca None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:53:49.447907 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95de6bf1-882f-400a-b802-2ade7ad0beca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:53:49.466421 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-95de6bf1-882f-400a-b802-2ade7ad0beca None None] MFA Rules not processed for user `ca155a48e2024c32af3fd2e19e5fac11`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 10:53:49.467614 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-95de6bf1-882f-400a-b802-2ade7ad0beca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 10:53:49.533856 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 587/1168] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 26 10:53:49 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 26 10:53:49.802484 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4112f69f-e8a7-4295-84ac-c657f2b271c5 None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:53:49.860776 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:53:49.903372 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4112f69f-e8a7-4295-84ac-c657f2b271c5 None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:53:49.935858 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65523465-b5c2-4ea8-a40c-3cc4141177e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:53:49.936453 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65523465-b5c2-4ea8-a40c-3cc4141177e0 None admin] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:53:49.936726 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65523465-b5c2-4ea8-a40c-3cc4141177e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:53:49.937567 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65523465-b5c2-4ea8-a40c-3cc4141177e0 None admin] PATH_INFO: `/v3/domains` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:53:49.938234 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65523465-b5c2-4ea8-a40c-3cc4141177e0 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:53:49.939577 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65523465-b5c2-4ea8-a40c-3cc4141177e0 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:53:49.954769 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 582/1169] 10.222.0.40 () {60 vars in 1138 bytes} [Fri Aug 26 10:53:49 2022] POST /identity/v3/domains => generated 267 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 10:53:51.878583 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e9a2e4-c27e-4ac4-ae2e-db27ca0fcf01 None None] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:53:51.879060 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e9a2e4-c27e-4ac4-ae2e-db27ca0fcf01 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:53:51.879161 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e9a2e4-c27e-4ac4-ae2e-db27ca0fcf01 None None] PATH_INFO: `/` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:53:51.880474 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 588/1170] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 26 10:53:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 10:53:51.885674 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71dabd9f-4f25-47bb-9fc5-6cbc4e97d179 None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:53:51.886121 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71dabd9f-4f25-47bb-9fc5-6cbc4e97d179 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:53:51.886264 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71dabd9f-4f25-47bb-9fc5-6cbc4e97d179 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:53:51.905056 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-71dabd9f-4f25-47bb-9fc5-6cbc4e97d179 None None] MFA Rules not processed for user `ca155a48e2024c32af3fd2e19e5fac11`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 10:53:51.906942 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-71dabd9f-4f25-47bb-9fc5-6cbc4e97d179 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 10:53:51.971183 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 583/1171] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 26 10:53:51 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 26 10:53:52.249422 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e9a2e4-c27e-4ac4-ae2e-db27ca0fcf01 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:53:52.264049 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:53:52.349967 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e9a2e4-c27e-4ac4-ae2e-db27ca0fcf01 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:53:52.383560 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52efd06f-961d-4c34-a3ef-2731c264cec2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:53:52.384179 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52efd06f-961d-4c34-a3ef-2731c264cec2 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:53:52.384451 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52efd06f-961d-4c34-a3ef-2731c264cec2 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:53:52.384721 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52efd06f-961d-4c34-a3ef-2731c264cec2 None admin] PATH_INFO: `/v3/domains/ca17b5a28d6b4fd6b35676c475129aee` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:53:52.395380 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52efd06f-961d-4c34-a3ef-2731c264cec2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=ca17b5a28d6b4fd6b35676c475129aee)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:53:52.396812 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52efd06f-961d-4c34-a3ef-2731c264cec2 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:53:52.398457 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 589/1172] 10.222.0.40 () {56 vars in 1183 bytes} [Fri Aug 26 10:53:52 2022] GET /identity/v3/domains/ca17b5a28d6b4fd6b35676c475129aee => generated 267 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 10:53:52.404255 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71dabd9f-4f25-47bb-9fc5-6cbc4e97d179 None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:53:52.451551 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:53:52.502237 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71dabd9f-4f25-47bb-9fc5-6cbc4e97d179 None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:53:52.536121 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898716d7-6003-442b-8e3f-059961bcfbd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:53:52.536707 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898716d7-6003-442b-8e3f-059961bcfbd2 None admin] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:53:52.536981 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898716d7-6003-442b-8e3f-059961bcfbd2 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:53:52.537249 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898716d7-6003-442b-8e3f-059961bcfbd2 None admin] PATH_INFO: `/v3/projects` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:53:52.537877 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-898716d7-6003-442b-8e3f-059961bcfbd2 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:53:52.539324 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-898716d7-6003-442b-8e3f-059961bcfbd2 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:53:52.565541 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 584/1173] 10.222.0.40 () {60 vars in 1141 bytes} [Fri Aug 26 10:53:52 2022] POST /identity/v3/projects => generated 361 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 10:53:54.486359 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59dbfff0-3a9d-46f7-be57-675673235334 None None] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:53:54.486913 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59dbfff0-3a9d-46f7-be57-675673235334 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:53:54.486913 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59dbfff0-3a9d-46f7-be57-675673235334 None None] PATH_INFO: `/` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:53:54.488232 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 590/1174] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 26 10:53:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 10:53:54.493349 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175a7559-94c5-4c9a-802c-62a3174740a0 None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:53:54.493729 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175a7559-94c5-4c9a-802c-62a3174740a0 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:53:54.493901 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175a7559-94c5-4c9a-802c-62a3174740a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:53:54.512139 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-175a7559-94c5-4c9a-802c-62a3174740a0 None None] MFA Rules not processed for user `ca155a48e2024c32af3fd2e19e5fac11`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 10:53:54.513318 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-175a7559-94c5-4c9a-802c-62a3174740a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 10:53:54.577603 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 585/1175] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 26 10:53:54 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 26 10:53:54.849326 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59dbfff0-3a9d-46f7-be57-675673235334 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:53:54.917648 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:53:54.947900 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59dbfff0-3a9d-46f7-be57-675673235334 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:53:54.980424 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5154fac9-1274-44c8-905a-d5f04ad7be84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:53:54.981044 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5154fac9-1274-44c8-905a-d5f04ad7be84 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:53:54.981324 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5154fac9-1274-44c8-905a-d5f04ad7be84 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:53:54.981598 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5154fac9-1274-44c8-905a-d5f04ad7be84 None admin] PATH_INFO: `/v3/domains/ca17b5a28d6b4fd6b35676c475129aee` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:53:54.982729 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5154fac9-1274-44c8-905a-d5f04ad7be84 None admin] RBAC: Authorizing `identity:get_domain(domain_id=ca17b5a28d6b4fd6b35676c475129aee)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:53:54.985048 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5154fac9-1274-44c8-905a-d5f04ad7be84 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:53:54.986640 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 591/1176] 10.222.0.40 () {56 vars in 1183 bytes} [Fri Aug 26 10:53:54 2022] GET /identity/v3/domains/ca17b5a28d6b4fd6b35676c475129aee => generated 267 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 10:53:54.992299 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175a7559-94c5-4c9a-802c-62a3174740a0 None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:53:55.021298 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:53:55.090036 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175a7559-94c5-4c9a-802c-62a3174740a0 None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:53:55.124732 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea60580-8706-426a-9b06-2d112d92d6ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:53:55.125327 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea60580-8706-426a-9b06-2d112d92d6ff None admin] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:53:55.125599 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea60580-8706-426a-9b06-2d112d92d6ff None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:53:55.125866 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea60580-8706-426a-9b06-2d112d92d6ff None admin] PATH_INFO: `/v3/users` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:53:55.126534 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea60580-8706-426a-9b06-2d112d92d6ff None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:53:55.127945 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea60580-8706-426a-9b06-2d112d92d6ff None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:53:55.147873 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 586/1177] 10.222.0.40 () {60 vars in 1132 bytes} [Fri Aug 26 10:53:54 2022] POST /identity/v3/users => generated 311 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 10:53:57.080806 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d651db-af8d-4c8d-86d5-352cc8cf0bfc None None] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:53:57.081343 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d651db-af8d-4c8d-86d5-352cc8cf0bfc None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:53:57.081343 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d651db-af8d-4c8d-86d5-352cc8cf0bfc None None] PATH_INFO: `/` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:53:57.082727 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 592/1178] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 26 10:53:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 10:53:57.087842 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e454784-c969-4596-b2bf-94c28d6ce879 None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:53:57.088212 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e454784-c969-4596-b2bf-94c28d6ce879 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:53:57.088381 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e454784-c969-4596-b2bf-94c28d6ce879 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:53:57.107065 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-3e454784-c969-4596-b2bf-94c28d6ce879 None None] MFA Rules not processed for user `ca155a48e2024c32af3fd2e19e5fac11`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 10:53:57.108243 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-3e454784-c969-4596-b2bf-94c28d6ce879 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 10:53:57.175089 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 587/1179] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 26 10:53:57 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 26 10:53:57.442908 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d651db-af8d-4c8d-86d5-352cc8cf0bfc None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:53:57.541532 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d651db-af8d-4c8d-86d5-352cc8cf0bfc None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:53:57.574650 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358acda4-e1e7-4c3e-a8dc-2c9f4c4cacf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:53:57.575264 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358acda4-e1e7-4c3e-a8dc-2c9f4c4cacf2 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:53:57.575539 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358acda4-e1e7-4c3e-a8dc-2c9f4c4cacf2 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:53:57.575807 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358acda4-e1e7-4c3e-a8dc-2c9f4c4cacf2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:53:57.583099 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:53:57.586240 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-358acda4-e1e7-4c3e-a8dc-2c9f4c4cacf2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 10:53:57.591077 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application [None req-358acda4-e1e7-4c3e-a8dc-2c9f4c4cacf2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 26 10:53:57.591077 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 10:53:57.591077 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 10:53:57.591077 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 10:53:57.591077 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 10:53:57.591077 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 10:53:57.591077 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 10:53:57.591077 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 10:53:57.591077 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 10:53:57.591077 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 10:53:57.591077 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 10:53:57.591077 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 10:53:57.591077 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 10:53:57.591077 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 10:53:57.591077 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 10:53:57.591077 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 10:53:57.592213 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 10:53:57.592213 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 10:53:57.592213 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 10:53:57.592213 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 10:53:57.592213 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 10:53:57.592213 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 10:53:57.592213 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 10:53:57.592213 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 10:53:57.592213 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 10:53:57.592213 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 10:53:57.592213 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 10:53:57.592213 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application with Lock( Aug 26 10:53:57.592213 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 10:53:57.592213 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._enter() Aug 26 10:53:57.592213 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 10:53:57.592213 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 10:53:57.593891 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 10:53:57.593891 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.creator() Aug 26 10:53:57.593891 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 10:53:57.593891 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application created_value = creator( Aug 26 10:53:57.593891 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 10:53:57.593891 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 10:53:57.593891 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 10:53:57.593891 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 10:53:57.593891 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 10:53:57.593891 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 10:53:57.593891 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 26 10:53:57.593891 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Aug 26 10:53:57.593891 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 593/1180] 10.222.0.40 () {56 vars in 1096 bytes} [Fri Aug 26 10:53:57 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 10:53:57.598228 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e454784-c969-4596-b2bf-94c28d6ce879 None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:53:57.615980 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:53:57.697387 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e454784-c969-4596-b2bf-94c28d6ce879 None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:53:57.729714 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864e1491-4ed9-4f1b-aaaf-b1420f5bcc3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:53:57.730327 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864e1491-4ed9-4f1b-aaaf-b1420f5bcc3f None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:53:57.730601 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864e1491-4ed9-4f1b-aaaf-b1420f5bcc3f None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:53:57.730865 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864e1491-4ed9-4f1b-aaaf-b1420f5bcc3f None admin] PATH_INFO: `/v3/roles` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:53:57.731516 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-864e1491-4ed9-4f1b-aaaf-b1420f5bcc3f None admin] RBAC: Adding query filter params (name=admin) {{(pid=81423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 10:53:57.731809 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-864e1491-4ed9-4f1b-aaaf-b1420f5bcc3f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:53:57.733155 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-864e1491-4ed9-4f1b-aaaf-b1420f5bcc3f None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:53:57.741968 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 588/1181] 10.222.0.40 () {56 vars in 1099 bytes} [Fri Aug 26 10:53:57 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 10:53:57.747676 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358acda4-e1e7-4c3e-a8dc-2c9f4c4cacf2 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:53:57.846085 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358acda4-e1e7-4c3e-a8dc-2c9f4c4cacf2 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:53:57.879873 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97759788-7fab-4a05-8d84-0e6efc5424ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:53:57.880492 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97759788-7fab-4a05-8d84-0e6efc5424ad None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:53:57.880767 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97759788-7fab-4a05-8d84-0e6efc5424ad None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:53:57.881036 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97759788-7fab-4a05-8d84-0e6efc5424ad None admin] PATH_INFO: `/v3/users/bac6a288cfec4b8aa0d4f94ea478a059` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:53:57.894337 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97759788-7fab-4a05-8d84-0e6efc5424ad None admin] RBAC: Authorizing `identity:get_user(user_id=bac6a288cfec4b8aa0d4f94ea478a059)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:53:57.895761 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97759788-7fab-4a05-8d84-0e6efc5424ad None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:53:57.897325 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 594/1182] 10.222.0.40 () {56 vars in 1177 bytes} [Fri Aug 26 10:53:57 2022] GET /identity/v3/users/bac6a288cfec4b8aa0d4f94ea478a059 => generated 311 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 10:53:57.903374 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864e1491-4ed9-4f1b-aaaf-b1420f5bcc3f None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:53:57.981390 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:53:58.003553 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864e1491-4ed9-4f1b-aaaf-b1420f5bcc3f None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:53:58.036055 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437766b6-2eb3-4ac3-8b6f-3a7c38a8db72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:53:58.036646 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437766b6-2eb3-4ac3-8b6f-3a7c38a8db72 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:53:58.036920 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437766b6-2eb3-4ac3-8b6f-3a7c38a8db72 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:53:58.037189 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437766b6-2eb3-4ac3-8b6f-3a7c38a8db72 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:53:58.102675 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-437766b6-2eb3-4ac3-8b6f-3a7c38a8db72 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:53:58.104110 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-437766b6-2eb3-4ac3-8b6f-3a7c38a8db72 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:53:58.180490 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:53:58.201763 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 589/1183] 10.222.0.40 () {58 vars in 1282 bytes} [Fri Aug 26 10:53:57 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 26 10:53:58.208349 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97759788-7fab-4a05-8d84-0e6efc5424ad None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:53:58.266970 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:53:58.308404 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97759788-7fab-4a05-8d84-0e6efc5424ad None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:53:58.341582 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca7abf5-482b-4aa6-990d-5f3bc0fd5664 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:53:58.342226 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca7abf5-482b-4aa6-990d-5f3bc0fd5664 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:53:58.342505 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca7abf5-482b-4aa6-990d-5f3bc0fd5664 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:53:58.342780 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca7abf5-482b-4aa6-990d-5f3bc0fd5664 None admin] PATH_INFO: `/v3/projects/a9e95831b7944ae3929a4051ba547326` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:53:58.344571 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca7abf5-482b-4aa6-990d-5f3bc0fd5664 None admin] RBAC: Authorizing `identity:get_project(project_id=a9e95831b7944ae3929a4051ba547326)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:53:58.346001 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca7abf5-482b-4aa6-990d-5f3bc0fd5664 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:53:58.347613 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 595/1184] 10.222.0.40 () {56 vars in 1186 bytes} [Fri Aug 26 10:53:58 2022] GET /identity/v3/projects/a9e95831b7944ae3929a4051ba547326 => generated 361 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 10:53:58.353755 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437766b6-2eb3-4ac3-8b6f-3a7c38a8db72 None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:53:58.448842 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437766b6-2eb3-4ac3-8b6f-3a7c38a8db72 None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:53:58.482249 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d471db88-ecd9-4067-aa26-27f167509341 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:53:58.482829 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d471db88-ecd9-4067-aa26-27f167509341 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:53:58.483101 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d471db88-ecd9-4067-aa26-27f167509341 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:53:58.483367 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d471db88-ecd9-4067-aa26-27f167509341 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:53:58.484207 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d471db88-ecd9-4067-aa26-27f167509341 None admin] RBAC: Adding query filter params (role.id=1f97e4602e2e4d6498913a81f9ae8d6a, scope.project.id=a9e95831b7944ae3929a4051ba547326, user.id=bac6a288cfec4b8aa0d4f94ea478a059) {{(pid=81423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 10:53:58.484757 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d471db88-ecd9-4067-aa26-27f167509341 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:53:58.486190 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d471db88-ecd9-4067-aa26-27f167509341 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:53:58.491359 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 590/1185] 10.222.0.40 () {56 vars in 1374 bytes} [Fri Aug 26 10:53:58 2022] GET /identity/v3/role_assignments?user.id=bac6a288cfec4b8aa0d4f94ea478a059&scope.project.id=a9e95831b7944ae3929a4051ba547326&role.id=1f97e4602e2e4d6498913a81f9ae8d6a => generated 259 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 10:54:00.374981 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befb0470-5b88-4bfc-949a-c5efaacbe4c0 None None] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:54:00.375517 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befb0470-5b88-4bfc-949a-c5efaacbe4c0 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:54:00.375517 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befb0470-5b88-4bfc-949a-c5efaacbe4c0 None None] PATH_INFO: `/` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:54:00.376836 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 596/1186] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 26 10:54:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 10:54:00.381879 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5cd6b2-ef70-4ee4-be28-de68c5f24819 None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:54:00.382319 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5cd6b2-ef70-4ee4-be28-de68c5f24819 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:54:00.382433 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5cd6b2-ef70-4ee4-be28-de68c5f24819 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:54:00.399926 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-ff5cd6b2-ef70-4ee4-be28-de68c5f24819 None None] MFA Rules not processed for user `ca155a48e2024c32af3fd2e19e5fac11`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 10:54:00.401094 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-ff5cd6b2-ef70-4ee4-be28-de68c5f24819 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 10:54:00.435831 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:54:00.470808 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 591/1187] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 26 10:54:00 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 26 10:54:00.742941 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-befb0470-5b88-4bfc-949a-c5efaacbe4c0 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:54:00.842663 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-befb0470-5b88-4bfc-949a-c5efaacbe4c0 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:54:00.867679 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:54:00.877491 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b30e1e1-7bcc-4dfc-98dc-c4627142072c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:54:00.878163 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b30e1e1-7bcc-4dfc-98dc-c4627142072c None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:54:00.878445 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b30e1e1-7bcc-4dfc-98dc-c4627142072c None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:54:00.878714 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b30e1e1-7bcc-4dfc-98dc-c4627142072c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:54:00.888644 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b30e1e1-7bcc-4dfc-98dc-c4627142072c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 10:54:00.895136 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application [None req-5b30e1e1-7bcc-4dfc-98dc-c4627142072c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 26 10:54:00.895136 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 10:54:00.895136 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 10:54:00.895136 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 10:54:00.895136 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 10:54:00.895136 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 10:54:00.895136 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 10:54:00.895136 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 10:54:00.895136 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 10:54:00.895136 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 10:54:00.895136 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 10:54:00.895136 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 10:54:00.895136 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 10:54:00.895136 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 10:54:00.895136 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 10:54:00.895136 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 10:54:00.898035 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 10:54:00.898035 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 10:54:00.898035 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 10:54:00.898035 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 10:54:00.898035 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 10:54:00.898035 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 10:54:00.898035 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 10:54:00.898035 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 10:54:00.898035 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 10:54:00.898035 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 10:54:00.898035 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 10:54:00.898035 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application with Lock( Aug 26 10:54:00.898035 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 10:54:00.898035 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._enter() Aug 26 10:54:00.898035 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 10:54:00.898035 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 10:54:00.899144 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 10:54:00.899144 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.creator() Aug 26 10:54:00.899144 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 10:54:00.899144 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application created_value = creator( Aug 26 10:54:00.899144 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 10:54:00.899144 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 10:54:00.899144 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 10:54:00.899144 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 10:54:00.899144 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 10:54:00.899144 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 10:54:00.899144 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 26 10:54:00.899144 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Aug 26 10:54:00.899144 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 597/1188] 10.222.0.40 () {56 vars in 1096 bytes} [Fri Aug 26 10:54:00 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 10:54:00.902301 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5cd6b2-ef70-4ee4-be28-de68c5f24819 None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:54:00.999919 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5cd6b2-ef70-4ee4-be28-de68c5f24819 None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:54:01.032986 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df78f8d-ce99-413d-9038-2bbf33cc56c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:54:01.033577 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df78f8d-ce99-413d-9038-2bbf33cc56c2 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:54:01.033857 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df78f8d-ce99-413d-9038-2bbf33cc56c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:54:01.034158 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df78f8d-ce99-413d-9038-2bbf33cc56c2 None admin] PATH_INFO: `/v3/roles` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:54:01.034819 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df78f8d-ce99-413d-9038-2bbf33cc56c2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 10:54:01.035120 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df78f8d-ce99-413d-9038-2bbf33cc56c2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:54:01.036469 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df78f8d-ce99-413d-9038-2bbf33cc56c2 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:54:01.044290 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 592/1189] 10.222.0.40 () {56 vars in 1099 bytes} [Fri Aug 26 10:54:00 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 10:54:01.050027 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b30e1e1-7bcc-4dfc-98dc-c4627142072c None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:54:01.148289 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b30e1e1-7bcc-4dfc-98dc-c4627142072c None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:54:01.181737 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f5333b-e9f3-403b-9c7c-12417c192615 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:54:01.182453 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f5333b-e9f3-403b-9c7c-12417c192615 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:54:01.182734 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f5333b-e9f3-403b-9c7c-12417c192615 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:54:01.183007 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f5333b-e9f3-403b-9c7c-12417c192615 None admin] PATH_INFO: `/v3/users/bac6a288cfec4b8aa0d4f94ea478a059` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:54:01.184133 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f5333b-e9f3-403b-9c7c-12417c192615 None admin] RBAC: Authorizing `identity:get_user(user_id=bac6a288cfec4b8aa0d4f94ea478a059)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:54:01.185565 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f5333b-e9f3-403b-9c7c-12417c192615 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:54:01.187849 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 598/1190] 10.222.0.40 () {56 vars in 1177 bytes} [Fri Aug 26 10:54:01 2022] GET /identity/v3/users/bac6a288cfec4b8aa0d4f94ea478a059 => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 10:54:01.193545 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df78f8d-ce99-413d-9038-2bbf33cc56c2 None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:54:01.226402 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:54:01.290846 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df78f8d-ce99-413d-9038-2bbf33cc56c2 None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:54:01.323316 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc077c6-5dd3-4b3f-a0dc-891e58e25552 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:54:01.323925 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc077c6-5dd3-4b3f-a0dc-891e58e25552 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:54:01.324197 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc077c6-5dd3-4b3f-a0dc-891e58e25552 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:54:01.324461 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc077c6-5dd3-4b3f-a0dc-891e58e25552 None admin] PATH_INFO: `/v3/projects/a9e95831b7944ae3929a4051ba547326` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:54:01.325547 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc077c6-5dd3-4b3f-a0dc-891e58e25552 None admin] RBAC: Authorizing `identity:get_project(project_id=a9e95831b7944ae3929a4051ba547326)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:54:01.326965 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc077c6-5dd3-4b3f-a0dc-891e58e25552 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:54:01.329251 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 593/1191] 10.222.0.40 () {56 vars in 1186 bytes} [Fri Aug 26 10:54:01 2022] GET /identity/v3/projects/a9e95831b7944ae3929a4051ba547326 => generated 361 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 10:54:01.335086 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f5333b-e9f3-403b-9c7c-12417c192615 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:54:01.390310 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:54:01.433904 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f5333b-e9f3-403b-9c7c-12417c192615 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:54:01.466901 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ed1595-62f1-46e3-b16e-a0ff15f0dc91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:54:01.467548 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ed1595-62f1-46e3-b16e-a0ff15f0dc91 None admin] REQUEST_METHOD: `PUT` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:54:01.467872 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ed1595-62f1-46e3-b16e-a0ff15f0dc91 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:54:01.468335 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ed1595-62f1-46e3-b16e-a0ff15f0dc91 None admin] PATH_INFO: `/v3/projects/a9e95831b7944ae3929a4051ba547326/users/bac6a288cfec4b8aa0d4f94ea478a059/roles/1f97e4602e2e4d6498913a81f9ae8d6a` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:54:01.470434 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ed1595-62f1-46e3-b16e-a0ff15f0dc91 None admin] RBAC: Authorizing `identity:create_grant(project_id=a9e95831b7944ae3929a4051ba547326, user_id=bac6a288cfec4b8aa0d4f94ea478a059, role_id=1f97e4602e2e4d6498913a81f9ae8d6a)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:54:01.471869 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ed1595-62f1-46e3-b16e-a0ff15f0dc91 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:54:01.487966 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 599/1192] 10.222.0.40 () {58 vars in 1439 bytes} [Fri Aug 26 10:54:01 2022] PUT /identity/v3/projects/a9e95831b7944ae3929a4051ba547326/users/bac6a288cfec4b8aa0d4f94ea478a059/roles/1f97e4602e2e4d6498913a81f9ae8d6a => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 10:54:03.393020 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f23943-6ba8-4e82-90e3-afd1355a86d3 None None] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:54:03.393535 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f23943-6ba8-4e82-90e3-afd1355a86d3 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:54:03.393535 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f23943-6ba8-4e82-90e3-afd1355a86d3 None None] PATH_INFO: `/` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:54:03.394912 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 594/1193] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 26 10:54:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 10:54:03.400050 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a731a5d4-6e1b-44f5-b1c8-d9955f074cab None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:54:03.400470 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a731a5d4-6e1b-44f5-b1c8-d9955f074cab None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:54:03.400599 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a731a5d4-6e1b-44f5-b1c8-d9955f074cab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:54:03.418527 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-a731a5d4-6e1b-44f5-b1c8-d9955f074cab None None] MFA Rules not processed for user `ca155a48e2024c32af3fd2e19e5fac11`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 10:54:03.419775 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-a731a5d4-6e1b-44f5-b1c8-d9955f074cab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 10:54:03.485792 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 600/1194] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 26 10:54:03 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 26 10:54:03.759185 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f23943-6ba8-4e82-90e3-afd1355a86d3 None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:54:03.865368 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f23943-6ba8-4e82-90e3-afd1355a86d3 None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:54:03.866908 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:54:03.899774 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4482d0d0-4ea2-48c2-9217-5df02418aeae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:54:03.900387 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4482d0d0-4ea2-48c2-9217-5df02418aeae None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:54:03.900657 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4482d0d0-4ea2-48c2-9217-5df02418aeae None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:54:03.900920 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4482d0d0-4ea2-48c2-9217-5df02418aeae None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:54:03.911256 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4482d0d0-4ea2-48c2-9217-5df02418aeae None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 10:54:03.915893 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application [None req-4482d0d0-4ea2-48c2-9217-5df02418aeae None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 26 10:54:03.915893 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 10:54:03.915893 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 10:54:03.915893 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 10:54:03.915893 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 10:54:03.915893 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 10:54:03.915893 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 10:54:03.915893 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 10:54:03.915893 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 10:54:03.915893 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 10:54:03.915893 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 10:54:03.915893 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 10:54:03.915893 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 10:54:03.915893 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 10:54:03.915893 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 10:54:03.915893 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 10:54:03.917045 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 10:54:03.917045 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 10:54:03.917045 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 10:54:03.917045 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 10:54:03.917045 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 10:54:03.917045 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 10:54:03.917045 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 10:54:03.917045 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 10:54:03.917045 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 10:54:03.917045 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 10:54:03.917045 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 10:54:03.917045 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application with Lock( Aug 26 10:54:03.917045 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 10:54:03.917045 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._enter() Aug 26 10:54:03.917045 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 10:54:03.917045 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 10:54:03.917898 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 10:54:03.917898 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.creator() Aug 26 10:54:03.917898 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 10:54:03.917898 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application created_value = creator( Aug 26 10:54:03.917898 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 10:54:03.917898 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 10:54:03.917898 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 10:54:03.917898 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 10:54:03.917898 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 10:54:03.917898 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 10:54:03.917898 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 26 10:54:03.917898 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Aug 26 10:54:03.917898 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 595/1195] 10.222.0.40 () {56 vars in 1096 bytes} [Fri Aug 26 10:54:03 2022] GET /identity/v3/roles/admin => generated 83 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 10:54:03.923712 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a731a5d4-6e1b-44f5-b1c8-d9955f074cab None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:54:03.967295 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:54:04.021132 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a731a5d4-6e1b-44f5-b1c8-d9955f074cab None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:54:04.054834 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9da84d3-559c-4e5c-82a2-86393493a532 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:54:04.055431 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9da84d3-559c-4e5c-82a2-86393493a532 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:54:04.055708 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9da84d3-559c-4e5c-82a2-86393493a532 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:54:04.055978 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9da84d3-559c-4e5c-82a2-86393493a532 None admin] PATH_INFO: `/v3/roles` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:54:04.056631 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9da84d3-559c-4e5c-82a2-86393493a532 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 10:54:04.056928 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9da84d3-559c-4e5c-82a2-86393493a532 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:54:04.058306 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9da84d3-559c-4e5c-82a2-86393493a532 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:54:04.066136 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 601/1196] 10.222.0.40 () {56 vars in 1099 bytes} [Fri Aug 26 10:54:03 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 10:54:04.071875 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4482d0d0-4ea2-48c2-9217-5df02418aeae None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:54:04.176045 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4482d0d0-4ea2-48c2-9217-5df02418aeae None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:54:04.213215 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4a0039-3c2d-42da-a699-9d4ef515d56a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:54:04.213837 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4a0039-3c2d-42da-a699-9d4ef515d56a None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:54:04.214143 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4a0039-3c2d-42da-a699-9d4ef515d56a None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:54:04.214417 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4a0039-3c2d-42da-a699-9d4ef515d56a None admin] PATH_INFO: `/v3/users/bac6a288cfec4b8aa0d4f94ea478a059` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:54:04.215526 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc4a0039-3c2d-42da-a699-9d4ef515d56a None admin] RBAC: Authorizing `identity:get_user(user_id=bac6a288cfec4b8aa0d4f94ea478a059)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:54:04.216940 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc4a0039-3c2d-42da-a699-9d4ef515d56a None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:54:04.220006 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 596/1197] 10.222.0.40 () {56 vars in 1177 bytes} [Fri Aug 26 10:54:04 2022] GET /identity/v3/users/bac6a288cfec4b8aa0d4f94ea478a059 => generated 311 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 10:54:04.225852 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9da84d3-559c-4e5c-82a2-86393493a532 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:54:04.322029 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9da84d3-559c-4e5c-82a2-86393493a532 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:54:04.329044 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:54:04.355453 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9992792-1520-47ca-bcc6-058939136b58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:54:04.356040 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9992792-1520-47ca-bcc6-058939136b58 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:54:04.356311 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9992792-1520-47ca-bcc6-058939136b58 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:54:04.356576 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9992792-1520-47ca-bcc6-058939136b58 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:54:04.420003 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9992792-1520-47ca-bcc6-058939136b58 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:54:04.422189 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9992792-1520-47ca-bcc6-058939136b58 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:54:04.516342 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 602/1198] 10.222.0.40 () {58 vars in 1282 bytes} [Fri Aug 26 10:54:04 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 291 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 26 10:54:04.522230 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4a0039-3c2d-42da-a699-9d4ef515d56a None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:54:04.553486 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:54:04.621852 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4a0039-3c2d-42da-a699-9d4ef515d56a None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:54:04.654092 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649c0000-3397-4d23-9368-a6d0df2644ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:54:04.654709 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649c0000-3397-4d23-9368-a6d0df2644ac None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:54:04.654987 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649c0000-3397-4d23-9368-a6d0df2644ac None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:54:04.655803 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649c0000-3397-4d23-9368-a6d0df2644ac None admin] PATH_INFO: `/v3/projects/a9e95831b7944ae3929a4051ba547326` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:54:04.656924 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-649c0000-3397-4d23-9368-a6d0df2644ac None admin] RBAC: Authorizing `identity:get_project(project_id=a9e95831b7944ae3929a4051ba547326)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:54:04.658363 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-649c0000-3397-4d23-9368-a6d0df2644ac None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:54:04.659977 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 597/1199] 10.222.0.40 () {56 vars in 1186 bytes} [Fri Aug 26 10:54:04 2022] GET /identity/v3/projects/a9e95831b7944ae3929a4051ba547326 => generated 361 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 10:54:04.666171 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9992792-1520-47ca-bcc6-058939136b58 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:54:04.674888 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:54:04.763250 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9992792-1520-47ca-bcc6-058939136b58 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:54:04.795386 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0843fa4d-b1b7-4d9f-bcbe-60aa92d5b0cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:54:04.796669 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0843fa4d-b1b7-4d9f-bcbe-60aa92d5b0cb None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:54:04.797112 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0843fa4d-b1b7-4d9f-bcbe-60aa92d5b0cb None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:54:04.797388 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0843fa4d-b1b7-4d9f-bcbe-60aa92d5b0cb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:54:04.798138 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0843fa4d-b1b7-4d9f-bcbe-60aa92d5b0cb None admin] RBAC: Adding query filter params (role.id=1f97e4602e2e4d6498913a81f9ae8d6a, scope.project.id=a9e95831b7944ae3929a4051ba547326, user.id=bac6a288cfec4b8aa0d4f94ea478a059) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 10:54:04.798444 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0843fa4d-b1b7-4d9f-bcbe-60aa92d5b0cb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:54:04.799853 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0843fa4d-b1b7-4d9f-bcbe-60aa92d5b0cb None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:54:04.805355 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 603/1200] 10.222.0.40 () {56 vars in 1374 bytes} [Fri Aug 26 10:54:04 2022] GET /identity/v3/role_assignments?user.id=bac6a288cfec4b8aa0d4f94ea478a059&scope.project.id=a9e95831b7944ae3929a4051ba547326&role.id=1f97e4602e2e4d6498913a81f9ae8d6a => generated 608 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 10:54:14.649989 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=51 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:54:14.802861 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=65 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:55:00.545514 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21dd934e-02e2-454f-bf23-7e6304f095bd None None] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:55:00.546366 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21dd934e-02e2-454f-bf23-7e6304f095bd None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:55:00.546366 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21dd934e-02e2-454f-bf23-7e6304f095bd None None] PATH_INFO: `/` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:55:00.547752 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 598/1201] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 26 10:55:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 10:55:00.554044 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950edd69-5bce-4846-8da9-0992e6cbdc2d None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:55:00.554539 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950edd69-5bce-4846-8da9-0992e6cbdc2d None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:55:00.554637 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950edd69-5bce-4846-8da9-0992e6cbdc2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:55:00.578969 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-950edd69-5bce-4846-8da9-0992e6cbdc2d None None] MFA Rules not processed for user `ca155a48e2024c32af3fd2e19e5fac11`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 10:55:00.580604 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-950edd69-5bce-4846-8da9-0992e6cbdc2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 10:55:00.665629 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 604/1202] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 26 10:55:00 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 26 10:55:00.941963 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21dd934e-02e2-454f-bf23-7e6304f095bd None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:55:01.082233 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21dd934e-02e2-454f-bf23-7e6304f095bd None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:55:01.148302 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdabccca-e36f-45b3-99a7-25a323403b1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:55:01.148902 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdabccca-e36f-45b3-99a7-25a323403b1f None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:55:01.149187 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdabccca-e36f-45b3-99a7-25a323403b1f None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:55:01.149434 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdabccca-e36f-45b3-99a7-25a323403b1f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:55:01.162119 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdabccca-e36f-45b3-99a7-25a323403b1f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:55:01.166291 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdabccca-e36f-45b3-99a7-25a323403b1f None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:55:01.183314 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application [None req-cdabccca-e36f-45b3-99a7-25a323403b1f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 10:55:01.183314 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 10:55:01.183314 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 10:55:01.183314 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 10:55:01.183314 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 10:55:01.183314 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 10:55:01.183314 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 10:55:01.183314 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 10:55:01.183314 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 10:55:01.183314 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 10:55:01.183314 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 10:55:01.183314 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 10:55:01.183314 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 10:55:01.183314 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 10:55:01.183314 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 10:55:01.183314 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 10:55:01.184730 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 10:55:01.184730 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 10:55:01.184730 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 10:55:01.184730 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 10:55:01.184730 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 10:55:01.184730 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 10:55:01.184730 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 10:55:01.184730 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application with Lock( Aug 26 10:55:01.184730 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 10:55:01.184730 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._enter() Aug 26 10:55:01.184730 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 10:55:01.184730 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 10:55:01.184730 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 10:55:01.184730 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.creator() Aug 26 10:55:01.184730 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 10:55:01.184730 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application created_value = creator( Aug 26 10:55:01.184730 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 10:55:01.186143 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 10:55:01.186143 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 10:55:01.186143 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Aug 26 10:55:01.186143 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 599/1203] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Aug 26 10:55:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 10:55:01.195406 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950edd69-5bce-4846-8da9-0992e6cbdc2d None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:55:01.323513 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950edd69-5bce-4846-8da9-0992e6cbdc2d None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:55:01.363542 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:55:01.369636 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c33a6ea-ef3b-4b9a-b2d1-05d0d5dda530 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:55:01.370301 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c33a6ea-ef3b-4b9a-b2d1-05d0d5dda530 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:55:01.370576 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c33a6ea-ef3b-4b9a-b2d1-05d0d5dda530 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:55:01.370841 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c33a6ea-ef3b-4b9a-b2d1-05d0d5dda530 None admin] PATH_INFO: `/v3/domains` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:55:01.371476 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c33a6ea-ef3b-4b9a-b2d1-05d0d5dda530 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 10:55:01.371794 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c33a6ea-ef3b-4b9a-b2d1-05d0d5dda530 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:55:01.373290 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c33a6ea-ef3b-4b9a-b2d1-05d0d5dda530 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:55:01.385561 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 605/1204] 10.222.0.40 () {56 vars in 1109 bytes} [Fri Aug 26 10:55:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 10:55:01.392483 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdabccca-e36f-45b3-99a7-25a323403b1f None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:55:01.455851 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:55:01.526100 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdabccca-e36f-45b3-99a7-25a323403b1f None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:55:01.564559 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e106bf2e-a69d-475b-906b-e8eef633f0ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:55:01.565220 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e106bf2e-a69d-475b-906b-e8eef633f0ea None admin] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:55:01.565458 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e106bf2e-a69d-475b-906b-e8eef633f0ea None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:55:01.565710 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e106bf2e-a69d-475b-906b-e8eef633f0ea None admin] PATH_INFO: `/v3/users` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:55:01.566383 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e106bf2e-a69d-475b-906b-e8eef633f0ea None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:55:01.567806 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e106bf2e-a69d-475b-906b-e8eef633f0ea None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:55:01.598642 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 600/1205] 10.222.0.40 () {60 vars in 1132 bytes} [Fri Aug 26 10:55:01 2022] POST /identity/v3/users => generated 251 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 10:55:03.792825 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0baa284-f05c-46e1-bbd5-917a17c82abb None None] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:55:03.793390 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0baa284-f05c-46e1-bbd5-917a17c82abb None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:55:03.793390 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0baa284-f05c-46e1-bbd5-917a17c82abb None None] PATH_INFO: `/` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:55:03.794761 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 606/1206] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 26 10:55:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 10:55:03.801554 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8ed1d7-87c5-4545-bdcb-a2c8f419e756 None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:55:03.802115 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8ed1d7-87c5-4545-bdcb-a2c8f419e756 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:55:03.802407 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8ed1d7-87c5-4545-bdcb-a2c8f419e756 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:55:03.824954 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-ad8ed1d7-87c5-4545-bdcb-a2c8f419e756 None None] MFA Rules not processed for user `ca155a48e2024c32af3fd2e19e5fac11`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 10:55:03.826957 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-ad8ed1d7-87c5-4545-bdcb-a2c8f419e756 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 10:55:03.914447 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 601/1207] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 26 10:55:03 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 26 10:55:04.191428 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0baa284-f05c-46e1-bbd5-917a17c82abb None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:55:04.312368 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0baa284-f05c-46e1-bbd5-917a17c82abb None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:55:04.349401 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6df1b1-4ecb-45dd-a4d5-e9b060e1c2da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:55:04.350059 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6df1b1-4ecb-45dd-a4d5-e9b060e1c2da None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:55:04.350613 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6df1b1-4ecb-45dd-a4d5-e9b060e1c2da None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:55:04.350613 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6df1b1-4ecb-45dd-a4d5-e9b060e1c2da None admin] PATH_INFO: `/v3/roles/service` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:55:04.361596 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa6df1b1-4ecb-45dd-a4d5-e9b060e1c2da None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 10:55:04.366353 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application [None req-aa6df1b1-4ecb-45dd-a4d5-e9b060e1c2da None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 26 10:55:04.366353 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 10:55:04.366353 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 10:55:04.366353 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 10:55:04.366353 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 10:55:04.366353 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 10:55:04.366353 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 10:55:04.366353 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 10:55:04.366353 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 10:55:04.366353 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 10:55:04.366353 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 10:55:04.366353 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 10:55:04.366353 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 10:55:04.366353 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 10:55:04.366353 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 10:55:04.366353 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 10:55:04.367509 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 10:55:04.367509 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 10:55:04.367509 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 10:55:04.367509 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 10:55:04.367509 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 10:55:04.367509 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 10:55:04.367509 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 10:55:04.367509 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 10:55:04.367509 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 10:55:04.367509 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 10:55:04.367509 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 10:55:04.367509 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application with Lock( Aug 26 10:55:04.367509 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 10:55:04.367509 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._enter() Aug 26 10:55:04.367509 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 10:55:04.367509 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 10:55:04.368703 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 10:55:04.368703 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.creator() Aug 26 10:55:04.368703 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 10:55:04.368703 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application created_value = creator( Aug 26 10:55:04.368703 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 10:55:04.368703 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 10:55:04.368703 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 10:55:04.368703 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 10:55:04.368703 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 10:55:04.368703 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 10:55:04.368703 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 26 10:55:04.368703 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Aug 26 10:55:04.368703 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 607/1208] 10.222.0.40 () {56 vars in 1102 bytes} [Fri Aug 26 10:55:04 2022] GET /identity/v3/roles/service => generated 85 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 10:55:04.373440 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8ed1d7-87c5-4545-bdcb-a2c8f419e756 None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:55:04.420547 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:55:04.486245 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8ed1d7-87c5-4545-bdcb-a2c8f419e756 None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:55:04.523273 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c06be04-46d2-447a-8f23-97370e1e63ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:55:04.524368 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c06be04-46d2-447a-8f23-97370e1e63ae None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:55:04.524680 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c06be04-46d2-447a-8f23-97370e1e63ae None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:55:04.525008 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c06be04-46d2-447a-8f23-97370e1e63ae None admin] PATH_INFO: `/v3/roles` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:55:04.525703 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c06be04-46d2-447a-8f23-97370e1e63ae None admin] RBAC: Adding query filter params (name=service) {{(pid=81423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 10:55:04.526119 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c06be04-46d2-447a-8f23-97370e1e63ae None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:55:04.527586 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c06be04-46d2-447a-8f23-97370e1e63ae None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:55:04.541527 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 602/1209] 10.222.0.40 () {56 vars in 1103 bytes} [Fri Aug 26 10:55:04 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 10:55:04.547493 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6df1b1-4ecb-45dd-a4d5-e9b060e1c2da None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:55:04.609884 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:55:04.677043 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6df1b1-4ecb-45dd-a4d5-e9b060e1c2da None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:55:04.726304 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd00368-c4bf-4442-b0cf-14255b6077ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:55:04.727125 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd00368-c4bf-4442-b0cf-14255b6077ab None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:55:04.727537 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd00368-c4bf-4442-b0cf-14255b6077ab None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:55:04.727815 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd00368-c4bf-4442-b0cf-14255b6077ab None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:55:04.739804 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd00368-c4bf-4442-b0cf-14255b6077ab None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:55:04.741780 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd00368-c4bf-4442-b0cf-14255b6077ab None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:55:04.760292 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application [None req-bfd00368-c4bf-4442-b0cf-14255b6077ab None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 10:55:04.760292 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 10:55:04.760292 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 10:55:04.760292 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 10:55:04.760292 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 10:55:04.760292 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 10:55:04.760292 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 10:55:04.760292 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 10:55:04.760292 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 10:55:04.760292 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 10:55:04.760292 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 10:55:04.760292 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 10:55:04.760292 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 10:55:04.760292 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 10:55:04.760292 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 10:55:04.760292 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 10:55:04.762210 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 10:55:04.762210 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 10:55:04.762210 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 10:55:04.762210 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 10:55:04.762210 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 10:55:04.762210 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 10:55:04.762210 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 10:55:04.762210 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application with Lock( Aug 26 10:55:04.762210 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 10:55:04.762210 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._enter() Aug 26 10:55:04.762210 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 10:55:04.762210 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 10:55:04.762210 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 10:55:04.762210 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.creator() Aug 26 10:55:04.762210 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 10:55:04.762210 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application created_value = creator( Aug 26 10:55:04.762210 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 10:55:04.764386 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 10:55:04.764386 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 10:55:04.764386 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Aug 26 10:55:04.766130 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 608/1210] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Aug 26 10:55:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 10:55:04.768528 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c06be04-46d2-447a-8f23-97370e1e63ae None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:55:04.877037 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:55:04.881310 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c06be04-46d2-447a-8f23-97370e1e63ae None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:55:04.923800 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0bc407e-a880-4d80-a5c8-3201290c7d23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:55:04.924481 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0bc407e-a880-4d80-a5c8-3201290c7d23 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:55:04.924833 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0bc407e-a880-4d80-a5c8-3201290c7d23 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:55:04.925179 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0bc407e-a880-4d80-a5c8-3201290c7d23 None admin] PATH_INFO: `/v3/domains` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:55:04.925943 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0bc407e-a880-4d80-a5c8-3201290c7d23 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 10:55:04.926310 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0bc407e-a880-4d80-a5c8-3201290c7d23 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:55:04.927813 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0bc407e-a880-4d80-a5c8-3201290c7d23 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:55:04.947329 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 603/1211] 10.222.0.40 () {56 vars in 1109 bytes} [Fri Aug 26 10:55:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 10:55:04.948980 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd00368-c4bf-4442-b0cf-14255b6077ab None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:55:05.064961 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd00368-c4bf-4442-b0cf-14255b6077ab None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:55:05.066677 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:55:05.105374 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8ded5e-4b08-4135-9a61-3ee4afd5007e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:55:05.106097 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8ded5e-4b08-4135-9a61-3ee4afd5007e None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:55:05.106361 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8ded5e-4b08-4135-9a61-3ee4afd5007e None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:55:05.106633 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8ded5e-4b08-4135-9a61-3ee4afd5007e None admin] PATH_INFO: `/v3/users/placement` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:55:05.116772 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df8ded5e-4b08-4135-9a61-3ee4afd5007e None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:55:05.118271 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df8ded5e-4b08-4135-9a61-3ee4afd5007e None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:55:05.130489 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application [None req-df8ded5e-4b08-4135-9a61-3ee4afd5007e None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 26 10:55:05.130489 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 10:55:05.130489 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 10:55:05.130489 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 10:55:05.130489 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 10:55:05.130489 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 10:55:05.130489 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 10:55:05.130489 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 10:55:05.130489 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 10:55:05.130489 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 10:55:05.130489 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 10:55:05.130489 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 10:55:05.130489 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 26 10:55:05.130489 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 26 10:55:05.130489 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 26 10:55:05.130489 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 26 10:55:05.131574 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 10:55:05.131574 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 10:55:05.131574 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 26 10:55:05.131574 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 10:55:05.131574 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 26 10:55:05.131574 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 10:55:05.131574 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 10:55:05.131574 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 10:55:05.131574 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 10:55:05.131574 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 10:55:05.131574 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 10:55:05.131574 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application with Lock( Aug 26 10:55:05.131574 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 10:55:05.131574 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._enter() Aug 26 10:55:05.131574 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 10:55:05.131574 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 10:55:05.131574 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 10:55:05.131574 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.creator() Aug 26 10:55:05.140028 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 10:55:05.140028 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application created_value = creator( Aug 26 10:55:05.140028 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 26 10:55:05.140028 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 26 10:55:05.140028 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 26 10:55:05.140028 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 26 10:55:05.140028 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 26 10:55:05.140028 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 26 10:55:05.140028 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 26 10:55:05.140028 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Aug 26 10:55:05.140028 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 609/1212] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Aug 26 10:55:04 2022] GET /identity/v3/users/placement => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 26 10:55:05.140737 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0bc407e-a880-4d80-a5c8-3201290c7d23 None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:55:05.262420 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0bc407e-a880-4d80-a5c8-3201290c7d23 None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:55:05.309138 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555c0661-cabe-42d0-a4f5-287bfc3aaf69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:55:05.309900 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555c0661-cabe-42d0-a4f5-287bfc3aaf69 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:55:05.310192 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555c0661-cabe-42d0-a4f5-287bfc3aaf69 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:55:05.310923 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555c0661-cabe-42d0-a4f5-287bfc3aaf69 None admin] PATH_INFO: `/v3/users` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:55:05.312247 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-555c0661-cabe-42d0-a4f5-287bfc3aaf69 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 10:55:05.312247 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-555c0661-cabe-42d0-a4f5-287bfc3aaf69 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:55:05.313709 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-555c0661-cabe-42d0-a4f5-287bfc3aaf69 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:55:05.342447 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 604/1213] 10.222.0.40 () {56 vars in 1143 bytes} [Fri Aug 26 10:55:05 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 10:55:05.343761 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8ded5e-4b08-4135-9a61-3ee4afd5007e None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:55:05.466664 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8ded5e-4b08-4135-9a61-3ee4afd5007e None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:55:05.505298 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98b8d67-34a6-43c5-a512-5124a514204d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:55:05.506126 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98b8d67-34a6-43c5-a512-5124a514204d None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:55:05.506384 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98b8d67-34a6-43c5-a512-5124a514204d None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:55:05.506657 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98b8d67-34a6-43c5-a512-5124a514204d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:55:05.539286 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:55:05.583014 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c98b8d67-34a6-43c5-a512-5124a514204d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:55:05.584431 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c98b8d67-34a6-43c5-a512-5124a514204d None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:55:05.696011 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 610/1214] 10.222.0.40 () {58 vars in 1282 bytes} [Fri Aug 26 10:55:05 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 26 10:55:05.697363 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555c0661-cabe-42d0-a4f5-287bfc3aaf69 None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:55:05.719005 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:55:05.815214 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555c0661-cabe-42d0-a4f5-287bfc3aaf69 None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:55:05.851092 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4b074e-926a-4586-af84-55dfd373e632 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:55:05.851718 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4b074e-926a-4586-af84-55dfd373e632 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:55:05.852004 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4b074e-926a-4586-af84-55dfd373e632 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:55:05.852255 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4b074e-926a-4586-af84-55dfd373e632 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:55:05.867233 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba4b074e-926a-4586-af84-55dfd373e632 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:55:05.869188 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba4b074e-926a-4586-af84-55dfd373e632 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:55:05.897180 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application [None req-ba4b074e-926a-4586-af84-55dfd373e632 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 10:55:05.897180 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 10:55:05.897180 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 10:55:05.897180 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 10:55:05.897180 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 10:55:05.897180 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 10:55:05.897180 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 10:55:05.897180 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 10:55:05.897180 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 10:55:05.897180 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 10:55:05.897180 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 10:55:05.897180 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 10:55:05.897180 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 10:55:05.897180 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 10:55:05.897180 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 10:55:05.897180 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 10:55:05.903503 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 10:55:05.903503 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 10:55:05.903503 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 10:55:05.903503 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 10:55:05.903503 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 10:55:05.903503 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 10:55:05.903503 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 10:55:05.903503 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application with Lock( Aug 26 10:55:05.903503 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 10:55:05.903503 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._enter() Aug 26 10:55:05.903503 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 10:55:05.903503 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 10:55:05.903503 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 10:55:05.903503 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.creator() Aug 26 10:55:05.903503 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 10:55:05.903503 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application created_value = creator( Aug 26 10:55:05.903503 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 10:55:05.904499 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 10:55:05.904499 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 10:55:05.904499 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Aug 26 10:55:05.904499 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 605/1215] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Aug 26 10:55:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 10:55:05.904802 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98b8d67-34a6-43c5-a512-5124a514204d None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:55:05.970438 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:55:06.012587 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98b8d67-34a6-43c5-a512-5124a514204d None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:55:06.050645 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f1c124-8c57-42b1-8eda-3b7fc83c65e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:55:06.051281 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f1c124-8c57-42b1-8eda-3b7fc83c65e8 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:55:06.051558 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f1c124-8c57-42b1-8eda-3b7fc83c65e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:55:06.051827 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f1c124-8c57-42b1-8eda-3b7fc83c65e8 None admin] PATH_INFO: `/v3/domains` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:55:06.052468 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f1c124-8c57-42b1-8eda-3b7fc83c65e8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 10:55:06.052785 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f1c124-8c57-42b1-8eda-3b7fc83c65e8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:55:06.054178 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f1c124-8c57-42b1-8eda-3b7fc83c65e8 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:55:06.072594 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 611/1216] 10.222.0.40 () {56 vars in 1109 bytes} [Fri Aug 26 10:55:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 10:55:06.072777 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4b074e-926a-4586-af84-55dfd373e632 None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:55:06.160660 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:55:06.199973 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4b074e-926a-4586-af84-55dfd373e632 None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:55:06.238030 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da583efc-1601-4c4c-952e-c08522feb66e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:55:06.238640 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da583efc-1601-4c4c-952e-c08522feb66e None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:55:06.238921 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da583efc-1601-4c4c-952e-c08522feb66e None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:55:06.239186 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da583efc-1601-4c4c-952e-c08522feb66e None admin] PATH_INFO: `/v3/projects/service` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:55:06.249382 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da583efc-1601-4c4c-952e-c08522feb66e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:55:06.251868 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da583efc-1601-4c4c-952e-c08522feb66e None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:55:06.268509 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application [None req-da583efc-1601-4c4c-952e-c08522feb66e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 10:55:06.268509 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 10:55:06.268509 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 10:55:06.268509 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 10:55:06.268509 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 10:55:06.268509 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 10:55:06.268509 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 10:55:06.268509 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 10:55:06.268509 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 10:55:06.268509 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 10:55:06.268509 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 10:55:06.268509 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 10:55:06.268509 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 26 10:55:06.268509 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 26 10:55:06.268509 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 26 10:55:06.269505 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 26 10:55:06.269505 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 10:55:06.269505 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 10:55:06.269505 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 10:55:06.269505 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 10:55:06.269505 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 10:55:06.269505 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 10:55:06.269505 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 10:55:06.269505 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application with Lock( Aug 26 10:55:06.269505 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 10:55:06.269505 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._enter() Aug 26 10:55:06.269505 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 10:55:06.269505 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 10:55:06.269505 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 10:55:06.269505 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.creator() Aug 26 10:55:06.269505 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 10:55:06.269505 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application created_value = creator( Aug 26 10:55:06.271202 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 26 10:55:06.271202 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 26 10:55:06.271202 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 26 10:55:06.271202 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 26 10:55:06.271202 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 26 10:55:06.271202 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 26 10:55:06.271202 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 10:55:06.271202 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Aug 26 10:55:06.271202 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 606/1217] 10.222.0.40 () {56 vars in 1111 bytes} [Fri Aug 26 10:55:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 10:55:06.276698 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f1c124-8c57-42b1-8eda-3b7fc83c65e8 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:55:06.409140 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f1c124-8c57-42b1-8eda-3b7fc83c65e8 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:55:06.440588 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:55:06.461332 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653030dd-858a-4930-8af8-b7e4d3f163ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:55:06.462158 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653030dd-858a-4930-8af8-b7e4d3f163ca None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:55:06.462538 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653030dd-858a-4930-8af8-b7e4d3f163ca None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:55:06.462818 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653030dd-858a-4930-8af8-b7e4d3f163ca None admin] PATH_INFO: `/v3/projects` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:55:06.463748 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-653030dd-858a-4930-8af8-b7e4d3f163ca None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 10:55:06.464056 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-653030dd-858a-4930-8af8-b7e4d3f163ca None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:55:06.465751 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-653030dd-858a-4930-8af8-b7e4d3f163ca None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:55:06.483951 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 612/1218] 10.222.0.40 () {56 vars in 1148 bytes} [Fri Aug 26 10:55:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 10:55:06.490777 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da583efc-1601-4c4c-952e-c08522feb66e None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:55:06.601874 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da583efc-1601-4c4c-952e-c08522feb66e None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:55:06.632130 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:55:06.638437 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30715da-0da6-40b0-b3e5-2d9bcfa66bd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:55:06.639663 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30715da-0da6-40b0-b3e5-2d9bcfa66bd1 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:55:06.640011 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30715da-0da6-40b0-b3e5-2d9bcfa66bd1 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:55:06.640311 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30715da-0da6-40b0-b3e5-2d9bcfa66bd1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:55:06.641199 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a30715da-0da6-40b0-b3e5-2d9bcfa66bd1 None admin] RBAC: Adding query filter params (role.id=3bda770fcd5c46cdbfabcd7404a25e66, scope.project.id=0e30ccf0c1204727b786876c141dbcd2, user.id=f29c5dfed31f42a9b5cbc0cd72e9fdc5) {{(pid=81423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 10:55:06.641507 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a30715da-0da6-40b0-b3e5-2d9bcfa66bd1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:55:06.643099 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a30715da-0da6-40b0-b3e5-2d9bcfa66bd1 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:55:06.649661 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 607/1219] 10.222.0.40 () {56 vars in 1374 bytes} [Fri Aug 26 10:55:06 2022] GET /identity/v3/role_assignments?user.id=f29c5dfed31f42a9b5cbc0cd72e9fdc5&scope.project.id=0e30ccf0c1204727b786876c141dbcd2&role.id=3bda770fcd5c46cdbfabcd7404a25e66 => generated 259 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 10:55:08.603077 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3dc144-b01d-4875-857b-01722081bc8d None None] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:55:08.603696 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3dc144-b01d-4875-857b-01722081bc8d None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:55:08.603696 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3dc144-b01d-4875-857b-01722081bc8d None None] PATH_INFO: `/` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:55:08.604990 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 613/1220] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 26 10:55:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 10:55:08.610969 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd1f8ef-ae6b-4e43-92c5-ff44b0f709cb None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:55:08.611432 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd1f8ef-ae6b-4e43-92c5-ff44b0f709cb None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:55:08.611619 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd1f8ef-ae6b-4e43-92c5-ff44b0f709cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:55:08.629588 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-8fd1f8ef-ae6b-4e43-92c5-ff44b0f709cb None None] MFA Rules not processed for user `ca155a48e2024c32af3fd2e19e5fac11`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 10:55:08.630798 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-8fd1f8ef-ae6b-4e43-92c5-ff44b0f709cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 10:55:08.701459 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 608/1221] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 26 10:55:08 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 26 10:55:08.972993 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3dc144-b01d-4875-857b-01722081bc8d None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:55:09.078766 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3dc144-b01d-4875-857b-01722081bc8d None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:55:09.115133 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f2e6d3-77b7-49c2-8c15-c40fc5635487 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:55:09.115727 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f2e6d3-77b7-49c2-8c15-c40fc5635487 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:55:09.116008 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f2e6d3-77b7-49c2-8c15-c40fc5635487 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:55:09.116285 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f2e6d3-77b7-49c2-8c15-c40fc5635487 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:55:09.126516 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f2e6d3-77b7-49c2-8c15-c40fc5635487 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 10:55:09.131309 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application [None req-88f2e6d3-77b7-49c2-8c15-c40fc5635487 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 26 10:55:09.131309 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 10:55:09.131309 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 10:55:09.131309 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 10:55:09.131309 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 10:55:09.131309 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 10:55:09.131309 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 10:55:09.131309 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 10:55:09.131309 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 10:55:09.131309 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 10:55:09.131309 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 10:55:09.131309 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 10:55:09.131309 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 10:55:09.131309 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 10:55:09.131309 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 10:55:09.131309 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 10:55:09.132371 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 10:55:09.132371 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 10:55:09.132371 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 10:55:09.132371 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 10:55:09.132371 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 10:55:09.132371 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 10:55:09.132371 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 10:55:09.132371 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 10:55:09.132371 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 10:55:09.132371 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 10:55:09.132371 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 10:55:09.132371 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application with Lock( Aug 26 10:55:09.132371 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 10:55:09.132371 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._enter() Aug 26 10:55:09.132371 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 10:55:09.132371 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 10:55:09.133262 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 10:55:09.133262 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.creator() Aug 26 10:55:09.133262 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 10:55:09.133262 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application created_value = creator( Aug 26 10:55:09.133262 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 10:55:09.133262 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 10:55:09.133262 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 10:55:09.133262 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 10:55:09.133262 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 10:55:09.133262 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 10:55:09.133262 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 26 10:55:09.133262 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Aug 26 10:55:09.133262 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 614/1222] 10.222.0.40 () {56 vars in 1102 bytes} [Fri Aug 26 10:55:08 2022] GET /identity/v3/roles/service => generated 85 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 26 10:55:09.138503 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd1f8ef-ae6b-4e43-92c5-ff44b0f709cb None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:55:09.250578 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd1f8ef-ae6b-4e43-92c5-ff44b0f709cb None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:55:09.278572 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:55:09.292064 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383a8c2f-9c1c-4c9c-b8e2-5e206d05dc75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:55:09.292673 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383a8c2f-9c1c-4c9c-b8e2-5e206d05dc75 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:55:09.292948 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383a8c2f-9c1c-4c9c-b8e2-5e206d05dc75 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:55:09.293238 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383a8c2f-9c1c-4c9c-b8e2-5e206d05dc75 None admin] PATH_INFO: `/v3/roles` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:55:09.293954 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-383a8c2f-9c1c-4c9c-b8e2-5e206d05dc75 None admin] RBAC: Adding query filter params (name=service) {{(pid=81423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 10:55:09.294322 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-383a8c2f-9c1c-4c9c-b8e2-5e206d05dc75 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:55:09.296608 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-383a8c2f-9c1c-4c9c-b8e2-5e206d05dc75 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:55:09.306049 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 609/1223] 10.222.0.40 () {56 vars in 1103 bytes} [Fri Aug 26 10:55:09 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 10:55:09.312738 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f2e6d3-77b7-49c2-8c15-c40fc5635487 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:55:09.356927 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:55:09.427619 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f2e6d3-77b7-49c2-8c15-c40fc5635487 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:55:09.465804 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b009622e-7fff-49a3-85f0-be92fad81c9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:55:09.466509 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b009622e-7fff-49a3-85f0-be92fad81c9b None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:55:09.467447 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b009622e-7fff-49a3-85f0-be92fad81c9b None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:55:09.467911 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b009622e-7fff-49a3-85f0-be92fad81c9b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:55:09.479778 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b009622e-7fff-49a3-85f0-be92fad81c9b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:55:09.481211 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b009622e-7fff-49a3-85f0-be92fad81c9b None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:55:09.497439 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application [None req-b009622e-7fff-49a3-85f0-be92fad81c9b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 10:55:09.497439 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 10:55:09.497439 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 10:55:09.497439 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 10:55:09.497439 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 10:55:09.497439 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 10:55:09.497439 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 10:55:09.497439 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 10:55:09.497439 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 10:55:09.497439 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 10:55:09.497439 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 10:55:09.497439 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 10:55:09.497439 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 10:55:09.497439 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 10:55:09.497439 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 10:55:09.497439 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 10:55:09.498627 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 10:55:09.498627 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 10:55:09.498627 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 10:55:09.498627 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 10:55:09.498627 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 10:55:09.498627 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 10:55:09.498627 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 10:55:09.498627 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application with Lock( Aug 26 10:55:09.498627 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 10:55:09.498627 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._enter() Aug 26 10:55:09.498627 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 10:55:09.498627 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 10:55:09.498627 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 10:55:09.498627 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.creator() Aug 26 10:55:09.498627 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 10:55:09.498627 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application created_value = creator( Aug 26 10:55:09.498627 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 10:55:09.499860 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 10:55:09.499860 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 10:55:09.499860 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Aug 26 10:55:09.499860 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 615/1224] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Aug 26 10:55:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 10:55:09.505037 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383a8c2f-9c1c-4c9c-b8e2-5e206d05dc75 None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:55:09.619186 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383a8c2f-9c1c-4c9c-b8e2-5e206d05dc75 None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:55:09.658092 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9225cb0-36fc-448e-a1f4-167aa717fa6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:55:09.658706 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9225cb0-36fc-448e-a1f4-167aa717fa6a None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:55:09.658984 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9225cb0-36fc-448e-a1f4-167aa717fa6a None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:55:09.659253 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9225cb0-36fc-448e-a1f4-167aa717fa6a None admin] PATH_INFO: `/v3/domains` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:55:09.659922 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9225cb0-36fc-448e-a1f4-167aa717fa6a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 10:55:09.660255 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9225cb0-36fc-448e-a1f4-167aa717fa6a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:55:09.661774 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9225cb0-36fc-448e-a1f4-167aa717fa6a None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:55:09.676365 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 610/1225] 10.222.0.40 () {56 vars in 1109 bytes} [Fri Aug 26 10:55:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 10:55:09.682348 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b009622e-7fff-49a3-85f0-be92fad81c9b None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:55:09.783298 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:55:09.791642 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b009622e-7fff-49a3-85f0-be92fad81c9b None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:55:09.828869 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4228e873-9043-4f4a-bb7a-7509972556e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:55:09.829574 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4228e873-9043-4f4a-bb7a-7509972556e2 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:55:09.829916 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4228e873-9043-4f4a-bb7a-7509972556e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:55:09.830231 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4228e873-9043-4f4a-bb7a-7509972556e2 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:55:09.839088 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4228e873-9043-4f4a-bb7a-7509972556e2 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:55:09.840671 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4228e873-9043-4f4a-bb7a-7509972556e2 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:55:09.855424 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application [None req-4228e873-9043-4f4a-bb7a-7509972556e2 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 26 10:55:09.855424 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 10:55:09.855424 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 10:55:09.855424 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 10:55:09.855424 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 10:55:09.855424 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 10:55:09.855424 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 10:55:09.855424 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 10:55:09.855424 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 10:55:09.855424 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 10:55:09.855424 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 10:55:09.855424 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 10:55:09.855424 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 26 10:55:09.855424 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 26 10:55:09.855424 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 26 10:55:09.855424 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 26 10:55:09.856553 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 10:55:09.856553 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 10:55:09.856553 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 26 10:55:09.856553 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 10:55:09.856553 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 26 10:55:09.856553 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 10:55:09.856553 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 10:55:09.856553 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 10:55:09.856553 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 10:55:09.856553 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 10:55:09.856553 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 10:55:09.856553 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application with Lock( Aug 26 10:55:09.856553 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 10:55:09.856553 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._enter() Aug 26 10:55:09.856553 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 10:55:09.856553 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 10:55:09.856553 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 10:55:09.856553 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.creator() Aug 26 10:55:09.857618 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 10:55:09.857618 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application created_value = creator( Aug 26 10:55:09.857618 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 26 10:55:09.857618 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 26 10:55:09.857618 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 26 10:55:09.857618 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 26 10:55:09.857618 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 26 10:55:09.857618 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 26 10:55:09.857618 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 26 10:55:09.857618 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Aug 26 10:55:09.857618 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 616/1226] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Aug 26 10:55:09 2022] GET /identity/v3/users/placement => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 10:55:09.863073 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9225cb0-36fc-448e-a1f4-167aa717fa6a None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:55:09.911684 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:55:09.969358 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9225cb0-36fc-448e-a1f4-167aa717fa6a None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:55:10.011035 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc87ab9-3476-47aa-8352-eff989b5e9fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:55:10.011640 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc87ab9-3476-47aa-8352-eff989b5e9fb None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:55:10.011915 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc87ab9-3476-47aa-8352-eff989b5e9fb None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:55:10.012182 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc87ab9-3476-47aa-8352-eff989b5e9fb None admin] PATH_INFO: `/v3/users` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:55:10.012898 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc87ab9-3476-47aa-8352-eff989b5e9fb None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 10:55:10.013195 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc87ab9-3476-47aa-8352-eff989b5e9fb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:55:10.015232 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc87ab9-3476-47aa-8352-eff989b5e9fb None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:55:10.037386 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 611/1227] 10.222.0.40 () {56 vars in 1143 bytes} [Fri Aug 26 10:55:09 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 10:55:10.038813 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4228e873-9043-4f4a-bb7a-7509972556e2 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:55:10.146067 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4228e873-9043-4f4a-bb7a-7509972556e2 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:55:10.182203 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8909501c-7fdd-4a2e-9803-2681b85a3917 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:55:10.182828 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8909501c-7fdd-4a2e-9803-2681b85a3917 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:55:10.183105 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8909501c-7fdd-4a2e-9803-2681b85a3917 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:55:10.183373 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8909501c-7fdd-4a2e-9803-2681b85a3917 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:55:10.195807 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8909501c-7fdd-4a2e-9803-2681b85a3917 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:55:10.197252 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8909501c-7fdd-4a2e-9803-2681b85a3917 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:55:10.212020 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application [None req-8909501c-7fdd-4a2e-9803-2681b85a3917 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 10:55:10.212020 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 10:55:10.212020 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 10:55:10.212020 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 10:55:10.212020 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 10:55:10.212020 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 10:55:10.212020 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 10:55:10.212020 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 10:55:10.212020 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 10:55:10.212020 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 10:55:10.212020 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 10:55:10.212020 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 10:55:10.212020 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 10:55:10.212020 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 10:55:10.212020 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 10:55:10.212020 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 10:55:10.213042 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 10:55:10.213042 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 10:55:10.213042 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 10:55:10.213042 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 10:55:10.213042 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 10:55:10.213042 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 10:55:10.213042 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 10:55:10.213042 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application with Lock( Aug 26 10:55:10.213042 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 10:55:10.213042 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._enter() Aug 26 10:55:10.213042 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 10:55:10.213042 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 10:55:10.213042 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 10:55:10.213042 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.creator() Aug 26 10:55:10.213042 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 10:55:10.213042 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application created_value = creator( Aug 26 10:55:10.213042 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 10:55:10.213945 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 10:55:10.213945 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 10:55:10.213945 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Aug 26 10:55:10.213945 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 617/1228] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Aug 26 10:55:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 10:55:10.219509 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc87ab9-3476-47aa-8352-eff989b5e9fb None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:55:10.346804 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc87ab9-3476-47aa-8352-eff989b5e9fb None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:55:10.349296 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:55:10.385369 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8c4a52-187b-43e0-b498-91366c3f2de3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:55:10.386067 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8c4a52-187b-43e0-b498-91366c3f2de3 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:55:10.386340 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8c4a52-187b-43e0-b498-91366c3f2de3 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:55:10.386612 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8c4a52-187b-43e0-b498-91366c3f2de3 None admin] PATH_INFO: `/v3/domains` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:55:10.387242 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae8c4a52-187b-43e0-b498-91366c3f2de3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 10:55:10.387541 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae8c4a52-187b-43e0-b498-91366c3f2de3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:55:10.389122 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae8c4a52-187b-43e0-b498-91366c3f2de3 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:55:10.401244 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 612/1229] 10.222.0.40 () {56 vars in 1109 bytes} [Fri Aug 26 10:55:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 10:55:10.407274 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8909501c-7fdd-4a2e-9803-2681b85a3917 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:55:10.417799 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:55:10.531915 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8909501c-7fdd-4a2e-9803-2681b85a3917 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:55:10.569572 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c771d97c-dc35-4775-ab72-f4f7a670de0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:55:10.570337 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c771d97c-dc35-4775-ab72-f4f7a670de0d None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:55:10.570662 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c771d97c-dc35-4775-ab72-f4f7a670de0d None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:55:10.570954 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c771d97c-dc35-4775-ab72-f4f7a670de0d None admin] PATH_INFO: `/v3/projects/service` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:55:10.578594 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c771d97c-dc35-4775-ab72-f4f7a670de0d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:55:10.580761 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c771d97c-dc35-4775-ab72-f4f7a670de0d None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:55:10.594419 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application [None req-c771d97c-dc35-4775-ab72-f4f7a670de0d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 10:55:10.594419 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 10:55:10.594419 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 10:55:10.594419 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 10:55:10.594419 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 10:55:10.594419 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 10:55:10.594419 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 10:55:10.594419 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 10:55:10.594419 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 10:55:10.594419 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 10:55:10.594419 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 10:55:10.594419 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 10:55:10.594419 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 26 10:55:10.594419 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 26 10:55:10.594419 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 26 10:55:10.595572 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 26 10:55:10.595572 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 10:55:10.595572 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 10:55:10.595572 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 10:55:10.595572 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 10:55:10.595572 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 10:55:10.595572 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 10:55:10.595572 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 10:55:10.595572 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application with Lock( Aug 26 10:55:10.595572 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 10:55:10.595572 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._enter() Aug 26 10:55:10.595572 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 10:55:10.595572 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 10:55:10.595572 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 10:55:10.595572 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.creator() Aug 26 10:55:10.595572 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 10:55:10.595572 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application created_value = creator( Aug 26 10:55:10.596758 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 26 10:55:10.596758 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 26 10:55:10.596758 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 26 10:55:10.596758 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 26 10:55:10.596758 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 26 10:55:10.596758 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 26 10:55:10.596758 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 10:55:10.596758 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Aug 26 10:55:10.601412 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 618/1230] 10.222.0.40 () {56 vars in 1111 bytes} [Fri Aug 26 10:55:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 10:55:10.602000 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8c4a52-187b-43e0-b498-91366c3f2de3 None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:55:10.715785 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8c4a52-187b-43e0-b498-91366c3f2de3 None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:55:10.755189 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e0ec98-b2a3-49d8-80f9-931e48f70c27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:55:10.755796 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e0ec98-b2a3-49d8-80f9-931e48f70c27 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:55:10.756074 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e0ec98-b2a3-49d8-80f9-931e48f70c27 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:55:10.756339 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e0ec98-b2a3-49d8-80f9-931e48f70c27 None admin] PATH_INFO: `/v3/projects` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:55:10.757027 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e0ec98-b2a3-49d8-80f9-931e48f70c27 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 10:55:10.757327 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e0ec98-b2a3-49d8-80f9-931e48f70c27 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:55:10.758873 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e0ec98-b2a3-49d8-80f9-931e48f70c27 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:55:10.770964 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 613/1231] 10.222.0.40 () {56 vars in 1148 bytes} [Fri Aug 26 10:55:10 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 10:55:10.776876 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c771d97c-dc35-4775-ab72-f4f7a670de0d None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:55:10.847724 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:55:10.888126 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c771d97c-dc35-4775-ab72-f4f7a670de0d None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:55:10.923927 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe5396b-52bf-48db-bf55-c425a621a36b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:55:10.924583 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe5396b-52bf-48db-bf55-c425a621a36b None admin] REQUEST_METHOD: `PUT` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:55:10.924859 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe5396b-52bf-48db-bf55-c425a621a36b None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:55:10.925134 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe5396b-52bf-48db-bf55-c425a621a36b None admin] PATH_INFO: `/v3/projects/0e30ccf0c1204727b786876c141dbcd2/users/f29c5dfed31f42a9b5cbc0cd72e9fdc5/roles/3bda770fcd5c46cdbfabcd7404a25e66` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:55:10.940125 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe5396b-52bf-48db-bf55-c425a621a36b None admin] RBAC: Authorizing `identity:create_grant(project_id=0e30ccf0c1204727b786876c141dbcd2, user_id=f29c5dfed31f42a9b5cbc0cd72e9fdc5, role_id=3bda770fcd5c46cdbfabcd7404a25e66)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:55:10.941677 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe5396b-52bf-48db-bf55-c425a621a36b None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:55:10.955597 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 619/1232] 10.222.0.40 () {58 vars in 1439 bytes} [Fri Aug 26 10:55:10 2022] PUT /identity/v3/projects/0e30ccf0c1204727b786876c141dbcd2/users/f29c5dfed31f42a9b5cbc0cd72e9fdc5/roles/3bda770fcd5c46cdbfabcd7404a25e66 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 10:55:12.982758 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f01221-b04b-452d-9160-5baa81a4a967 None None] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:55:12.983358 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f01221-b04b-452d-9160-5baa81a4a967 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:55:12.983576 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f01221-b04b-452d-9160-5baa81a4a967 None None] PATH_INFO: `/` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:55:12.985022 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 614/1233] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 26 10:55:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 10:55:12.991307 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a7a079-8551-4c96-ad7f-85db853d6302 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:55:12.991779 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a7a079-8551-4c96-ad7f-85db853d6302 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:55:12.991871 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a7a079-8551-4c96-ad7f-85db853d6302 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:55:13.012221 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-e4a7a079-8551-4c96-ad7f-85db853d6302 None None] MFA Rules not processed for user `ca155a48e2024c32af3fd2e19e5fac11`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 10:55:13.014470 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-e4a7a079-8551-4c96-ad7f-85db853d6302 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 10:55:13.090230 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 620/1234] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 26 10:55:12 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 26 10:55:13.399375 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f01221-b04b-452d-9160-5baa81a4a967 None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:55:13.426563 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:55:13.512126 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f01221-b04b-452d-9160-5baa81a4a967 None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:55:13.549116 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b8bf2b-fe5a-4fc9-8571-b89b19b2b8b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:55:13.549740 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b8bf2b-fe5a-4fc9-8571-b89b19b2b8b0 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:55:13.550041 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b8bf2b-fe5a-4fc9-8571-b89b19b2b8b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:55:13.550314 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b8bf2b-fe5a-4fc9-8571-b89b19b2b8b0 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:55:13.560974 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b8bf2b-fe5a-4fc9-8571-b89b19b2b8b0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 10:55:13.566078 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application [None req-01b8bf2b-fe5a-4fc9-8571-b89b19b2b8b0 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 26 10:55:13.566078 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 10:55:13.566078 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 10:55:13.566078 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 10:55:13.566078 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 10:55:13.566078 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 10:55:13.566078 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 10:55:13.566078 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 10:55:13.566078 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 10:55:13.566078 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 10:55:13.566078 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 10:55:13.566078 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 10:55:13.566078 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 10:55:13.566078 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 10:55:13.566078 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 10:55:13.566078 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 10:55:13.567196 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 10:55:13.567196 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 10:55:13.567196 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 10:55:13.567196 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 10:55:13.567196 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 10:55:13.567196 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 10:55:13.567196 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 10:55:13.567196 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 10:55:13.567196 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 10:55:13.567196 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 10:55:13.567196 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 10:55:13.567196 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application with Lock( Aug 26 10:55:13.567196 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 10:55:13.567196 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._enter() Aug 26 10:55:13.567196 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 10:55:13.567196 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 10:55:13.571160 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 10:55:13.571160 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.creator() Aug 26 10:55:13.571160 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 10:55:13.571160 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application created_value = creator( Aug 26 10:55:13.571160 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 10:55:13.571160 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 10:55:13.571160 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 10:55:13.571160 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 10:55:13.571160 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 10:55:13.571160 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 10:55:13.571160 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 26 10:55:13.571160 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Aug 26 10:55:13.571160 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 615/1235] 10.222.0.40 () {56 vars in 1102 bytes} [Fri Aug 26 10:55:13 2022] GET /identity/v3/roles/service => generated 85 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 10:55:13.573681 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a7a079-8551-4c96-ad7f-85db853d6302 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:55:13.625767 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:55:13.683769 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a7a079-8551-4c96-ad7f-85db853d6302 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:55:13.722289 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da10ca83-3a6f-4950-93d0-a5b9634f223c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:55:13.722927 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da10ca83-3a6f-4950-93d0-a5b9634f223c None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:55:13.723359 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da10ca83-3a6f-4950-93d0-a5b9634f223c None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:55:13.723667 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da10ca83-3a6f-4950-93d0-a5b9634f223c None admin] PATH_INFO: `/v3/roles` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:55:13.724483 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da10ca83-3a6f-4950-93d0-a5b9634f223c None admin] RBAC: Adding query filter params (name=service) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 10:55:13.724790 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da10ca83-3a6f-4950-93d0-a5b9634f223c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:55:13.726196 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da10ca83-3a6f-4950-93d0-a5b9634f223c None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:55:13.736571 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 621/1236] 10.222.0.40 () {56 vars in 1103 bytes} [Fri Aug 26 10:55:13 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 10:55:13.742177 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b8bf2b-fe5a-4fc9-8571-b89b19b2b8b0 None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:55:13.838189 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:55:13.856560 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b8bf2b-fe5a-4fc9-8571-b89b19b2b8b0 None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:55:13.900381 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb4cd23-e2d0-4d7f-b0df-9df0d20e3070 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:55:13.901173 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb4cd23-e2d0-4d7f-b0df-9df0d20e3070 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:55:13.901498 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb4cd23-e2d0-4d7f-b0df-9df0d20e3070 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:55:13.901771 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb4cd23-e2d0-4d7f-b0df-9df0d20e3070 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:55:13.913751 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb4cd23-e2d0-4d7f-b0df-9df0d20e3070 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:55:13.916023 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb4cd23-e2d0-4d7f-b0df-9df0d20e3070 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:55:13.932954 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application [None req-0eb4cd23-e2d0-4d7f-b0df-9df0d20e3070 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 10:55:13.932954 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 10:55:13.932954 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 10:55:13.932954 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 10:55:13.932954 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 10:55:13.932954 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 10:55:13.932954 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 10:55:13.932954 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 10:55:13.932954 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 10:55:13.932954 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 10:55:13.932954 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 10:55:13.932954 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 10:55:13.932954 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 10:55:13.932954 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 10:55:13.932954 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 10:55:13.932954 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 10:55:13.934019 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 10:55:13.934019 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 10:55:13.934019 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 10:55:13.934019 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 10:55:13.934019 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 10:55:13.934019 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 10:55:13.934019 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 10:55:13.934019 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application with Lock( Aug 26 10:55:13.934019 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 10:55:13.934019 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._enter() Aug 26 10:55:13.934019 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 10:55:13.934019 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 10:55:13.934019 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 10:55:13.934019 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.creator() Aug 26 10:55:13.934019 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 10:55:13.934019 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application created_value = creator( Aug 26 10:55:13.934019 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 10:55:13.934935 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 10:55:13.934935 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 10:55:13.934935 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Aug 26 10:55:13.934935 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 616/1237] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Aug 26 10:55:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 10:55:13.940847 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da10ca83-3a6f-4950-93d0-a5b9634f223c None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:55:14.050613 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da10ca83-3a6f-4950-93d0-a5b9634f223c None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:55:14.059718 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:55:14.087532 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87097f1a-5a52-4c99-8c3c-15c66d4524c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:55:14.088198 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87097f1a-5a52-4c99-8c3c-15c66d4524c2 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:55:14.088479 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87097f1a-5a52-4c99-8c3c-15c66d4524c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:55:14.088780 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87097f1a-5a52-4c99-8c3c-15c66d4524c2 None admin] PATH_INFO: `/v3/domains` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:55:14.089425 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87097f1a-5a52-4c99-8c3c-15c66d4524c2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 10:55:14.089729 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87097f1a-5a52-4c99-8c3c-15c66d4524c2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:55:14.091115 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87097f1a-5a52-4c99-8c3c-15c66d4524c2 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:55:14.104530 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 622/1238] 10.222.0.40 () {56 vars in 1109 bytes} [Fri Aug 26 10:55:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 10:55:14.111094 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb4cd23-e2d0-4d7f-b0df-9df0d20e3070 None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:55:14.225589 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb4cd23-e2d0-4d7f-b0df-9df0d20e3070 None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:55:14.267134 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57dfc0c2-cc5e-4c40-b836-34151bb4b37c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:55:14.267786 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57dfc0c2-cc5e-4c40-b836-34151bb4b37c None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:55:14.268148 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57dfc0c2-cc5e-4c40-b836-34151bb4b37c None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:55:14.268330 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57dfc0c2-cc5e-4c40-b836-34151bb4b37c None admin] PATH_INFO: `/v3/users/placement` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:55:14.271722 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:55:14.277908 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57dfc0c2-cc5e-4c40-b836-34151bb4b37c None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:55:14.281128 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57dfc0c2-cc5e-4c40-b836-34151bb4b37c None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:55:14.295141 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application [None req-57dfc0c2-cc5e-4c40-b836-34151bb4b37c None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 26 10:55:14.295141 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 10:55:14.295141 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 10:55:14.295141 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 10:55:14.295141 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 10:55:14.295141 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 10:55:14.295141 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 10:55:14.295141 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 10:55:14.295141 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 10:55:14.295141 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 10:55:14.295141 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 10:55:14.295141 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 10:55:14.295141 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 26 10:55:14.295141 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 26 10:55:14.295141 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 26 10:55:14.295141 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 26 10:55:14.296573 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 10:55:14.296573 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 10:55:14.296573 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 26 10:55:14.296573 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 10:55:14.296573 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 26 10:55:14.296573 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 10:55:14.296573 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 10:55:14.296573 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 10:55:14.296573 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 10:55:14.296573 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 10:55:14.296573 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 10:55:14.296573 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application with Lock( Aug 26 10:55:14.296573 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 10:55:14.296573 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._enter() Aug 26 10:55:14.296573 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 10:55:14.296573 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 10:55:14.296573 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 10:55:14.296573 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.creator() Aug 26 10:55:14.297598 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 10:55:14.297598 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application created_value = creator( Aug 26 10:55:14.297598 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 26 10:55:14.297598 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 26 10:55:14.297598 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 26 10:55:14.297598 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 26 10:55:14.297598 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 26 10:55:14.297598 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 26 10:55:14.297598 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 26 10:55:14.297598 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Aug 26 10:55:14.297598 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 617/1239] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Aug 26 10:55:14 2022] GET /identity/v3/users/placement => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 26 10:55:14.303237 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87097f1a-5a52-4c99-8c3c-15c66d4524c2 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:55:14.417651 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87097f1a-5a52-4c99-8c3c-15c66d4524c2 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:55:14.456507 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ba382f-a552-4ef6-af48-2ab4aeff45c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:55:14.457195 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ba382f-a552-4ef6-af48-2ab4aeff45c1 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:55:14.457488 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ba382f-a552-4ef6-af48-2ab4aeff45c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:55:14.457755 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ba382f-a552-4ef6-af48-2ab4aeff45c1 None admin] PATH_INFO: `/v3/users` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:55:14.458584 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7ba382f-a552-4ef6-af48-2ab4aeff45c1 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 10:55:14.458898 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7ba382f-a552-4ef6-af48-2ab4aeff45c1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:55:14.460585 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7ba382f-a552-4ef6-af48-2ab4aeff45c1 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:55:14.481293 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 623/1240] 10.222.0.40 () {56 vars in 1143 bytes} [Fri Aug 26 10:55:14 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 10:55:14.488048 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57dfc0c2-cc5e-4c40-b836-34151bb4b37c None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:55:14.601309 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57dfc0c2-cc5e-4c40-b836-34151bb4b37c None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:55:14.637098 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76886517-8068-41df-a586-8020e7ad3ac3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:55:14.637720 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76886517-8068-41df-a586-8020e7ad3ac3 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:55:14.637961 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76886517-8068-41df-a586-8020e7ad3ac3 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:55:14.638319 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76886517-8068-41df-a586-8020e7ad3ac3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:55:14.707719 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:55:14.714796 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76886517-8068-41df-a586-8020e7ad3ac3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:55:14.716416 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76886517-8068-41df-a586-8020e7ad3ac3 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:55:14.824212 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 618/1241] 10.222.0.40 () {58 vars in 1282 bytes} [Fri Aug 26 10:55:14 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 26 10:55:14.829963 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ba382f-a552-4ef6-af48-2ab4aeff45c1 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:55:14.860847 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:55:14.941014 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ba382f-a552-4ef6-af48-2ab4aeff45c1 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:55:14.981514 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7217b9-b66d-4584-96ef-b0d4462c6e20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:55:14.982738 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7217b9-b66d-4584-96ef-b0d4462c6e20 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:55:14.983134 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7217b9-b66d-4584-96ef-b0d4462c6e20 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:55:14.983416 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7217b9-b66d-4584-96ef-b0d4462c6e20 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:55:14.996572 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef7217b9-b66d-4584-96ef-b0d4462c6e20 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:55:14.998651 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef7217b9-b66d-4584-96ef-b0d4462c6e20 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:55:15.014250 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application [None req-ef7217b9-b66d-4584-96ef-b0d4462c6e20 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 10:55:15.014250 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 10:55:15.014250 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 10:55:15.014250 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 10:55:15.014250 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 10:55:15.014250 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 10:55:15.014250 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 10:55:15.014250 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 10:55:15.014250 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 10:55:15.014250 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 10:55:15.014250 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 10:55:15.014250 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 10:55:15.014250 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 10:55:15.014250 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 10:55:15.014250 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 10:55:15.014250 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 10:55:15.015274 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 10:55:15.015274 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 10:55:15.015274 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 10:55:15.015274 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 10:55:15.015274 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 10:55:15.015274 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 10:55:15.015274 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 10:55:15.015274 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application with Lock( Aug 26 10:55:15.015274 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 10:55:15.015274 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._enter() Aug 26 10:55:15.015274 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 10:55:15.015274 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 10:55:15.015274 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 10:55:15.015274 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.creator() Aug 26 10:55:15.015274 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 10:55:15.015274 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application created_value = creator( Aug 26 10:55:15.015274 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 10:55:15.016220 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 10:55:15.016220 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 10:55:15.016220 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Aug 26 10:55:15.016220 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 624/1242] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Aug 26 10:55:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 10:55:15.021226 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76886517-8068-41df-a586-8020e7ad3ac3 None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:55:15.129102 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:55:15.136883 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76886517-8068-41df-a586-8020e7ad3ac3 None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:55:15.175147 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1c4a18-586a-471d-b89f-65ef6c78a77b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:55:15.175813 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1c4a18-586a-471d-b89f-65ef6c78a77b None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:55:15.176093 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1c4a18-586a-471d-b89f-65ef6c78a77b None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:55:15.176400 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1c4a18-586a-471d-b89f-65ef6c78a77b None admin] PATH_INFO: `/v3/domains` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:55:15.177065 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b1c4a18-586a-471d-b89f-65ef6c78a77b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 10:55:15.177387 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b1c4a18-586a-471d-b89f-65ef6c78a77b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:55:15.178892 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b1c4a18-586a-471d-b89f-65ef6c78a77b None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:55:15.196713 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 619/1243] 10.222.0.40 () {56 vars in 1109 bytes} [Fri Aug 26 10:55:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 10:55:15.197908 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7217b9-b66d-4584-96ef-b0d4462c6e20 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:55:15.285547 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:55:15.312070 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7217b9-b66d-4584-96ef-b0d4462c6e20 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:55:15.347492 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec99710-ab51-4cab-93e5-c0ee24d53371 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:55:15.348116 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec99710-ab51-4cab-93e5-c0ee24d53371 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:55:15.348390 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec99710-ab51-4cab-93e5-c0ee24d53371 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:55:15.348654 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec99710-ab51-4cab-93e5-c0ee24d53371 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:55:15.355436 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec99710-ab51-4cab-93e5-c0ee24d53371 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:55:15.356900 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec99710-ab51-4cab-93e5-c0ee24d53371 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:55:15.369094 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application [None req-8ec99710-ab51-4cab-93e5-c0ee24d53371 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 10:55:15.369094 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 10:55:15.369094 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 10:55:15.369094 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 10:55:15.369094 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 10:55:15.369094 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 10:55:15.369094 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 10:55:15.369094 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 10:55:15.369094 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 10:55:15.369094 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 10:55:15.369094 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 10:55:15.369094 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 10:55:15.369094 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 26 10:55:15.369094 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 26 10:55:15.369094 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 26 10:55:15.370174 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 26 10:55:15.370174 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 10:55:15.370174 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 10:55:15.370174 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 10:55:15.370174 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 10:55:15.370174 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 10:55:15.370174 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 10:55:15.370174 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 10:55:15.370174 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application with Lock( Aug 26 10:55:15.370174 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 10:55:15.370174 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._enter() Aug 26 10:55:15.370174 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 10:55:15.370174 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 10:55:15.370174 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 10:55:15.370174 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.creator() Aug 26 10:55:15.370174 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 10:55:15.370174 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application created_value = creator( Aug 26 10:55:15.371101 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 26 10:55:15.371101 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 26 10:55:15.371101 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 26 10:55:15.371101 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 26 10:55:15.371101 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 26 10:55:15.371101 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 26 10:55:15.371101 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 10:55:15.371101 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Aug 26 10:55:15.371101 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 625/1244] 10.222.0.40 () {56 vars in 1111 bytes} [Fri Aug 26 10:55:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 26 10:55:15.376577 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1c4a18-586a-471d-b89f-65ef6c78a77b None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:55:15.486723 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1c4a18-586a-471d-b89f-65ef6c78a77b None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:55:15.524924 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72240c74-0935-4aa9-8ca1-356130b93738 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:55:15.525672 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72240c74-0935-4aa9-8ca1-356130b93738 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:55:15.526033 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72240c74-0935-4aa9-8ca1-356130b93738 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:55:15.526376 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72240c74-0935-4aa9-8ca1-356130b93738 None admin] PATH_INFO: `/v3/projects` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:55:15.527142 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72240c74-0935-4aa9-8ca1-356130b93738 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 10:55:15.527446 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72240c74-0935-4aa9-8ca1-356130b93738 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:55:15.529157 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72240c74-0935-4aa9-8ca1-356130b93738 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:55:15.546372 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 620/1245] 10.222.0.40 () {56 vars in 1148 bytes} [Fri Aug 26 10:55:15 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 10:55:15.552684 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec99710-ab51-4cab-93e5-c0ee24d53371 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:55:15.656598 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec99710-ab51-4cab-93e5-c0ee24d53371 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:55:15.691793 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d506a94e-d13f-445f-b2d5-79f8fbde6b2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:55:15.692490 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d506a94e-d13f-445f-b2d5-79f8fbde6b2a None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:55:15.692763 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d506a94e-d13f-445f-b2d5-79f8fbde6b2a None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:55:15.693032 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d506a94e-d13f-445f-b2d5-79f8fbde6b2a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:55:15.693753 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d506a94e-d13f-445f-b2d5-79f8fbde6b2a None admin] RBAC: Adding query filter params (role.id=3bda770fcd5c46cdbfabcd7404a25e66, scope.project.id=0e30ccf0c1204727b786876c141dbcd2, user.id=f29c5dfed31f42a9b5cbc0cd72e9fdc5) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 10:55:15.694080 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d506a94e-d13f-445f-b2d5-79f8fbde6b2a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:55:15.695498 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d506a94e-d13f-445f-b2d5-79f8fbde6b2a None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:55:15.702951 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 626/1246] 10.222.0.40 () {56 vars in 1374 bytes} [Fri Aug 26 10:55:15 2022] GET /identity/v3/role_assignments?user.id=f29c5dfed31f42a9b5cbc0cd72e9fdc5&scope.project.id=0e30ccf0c1204727b786876c141dbcd2&role.id=3bda770fcd5c46cdbfabcd7404a25e66 => generated 608 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 10:55:17.797017 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f171a54-97b4-4d8a-b34e-07542cc7de56 None None] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:55:17.797017 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f171a54-97b4-4d8a-b34e-07542cc7de56 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:55:17.797017 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f171a54-97b4-4d8a-b34e-07542cc7de56 None None] PATH_INFO: `/` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:55:17.797017 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 621/1247] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 26 10:55:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 10:55:17.802516 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e868578-e587-44a3-b44d-dd40b4670363 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:55:17.802933 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e868578-e587-44a3-b44d-dd40b4670363 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:55:17.803070 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e868578-e587-44a3-b44d-dd40b4670363 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:55:17.809439 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:55:17.823297 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-0e868578-e587-44a3-b44d-dd40b4670363 None None] MFA Rules not processed for user `ca155a48e2024c32af3fd2e19e5fac11`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 10:55:17.824529 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-0e868578-e587-44a3-b44d-dd40b4670363 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 10:55:17.903684 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 627/1248] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 26 10:55:17 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 26 10:55:18.172396 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f171a54-97b4-4d8a-b34e-07542cc7de56 None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:55:18.191291 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:55:18.285316 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f171a54-97b4-4d8a-b34e-07542cc7de56 None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:55:18.324803 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4c19b0-f7ad-4b19-a5df-a0913b4ae742 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:55:18.325426 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4c19b0-f7ad-4b19-a5df-a0913b4ae742 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:55:18.325701 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4c19b0-f7ad-4b19-a5df-a0913b4ae742 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:55:18.326604 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4c19b0-f7ad-4b19-a5df-a0913b4ae742 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:55:18.338331 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e4c19b0-f7ad-4b19-a5df-a0913b4ae742 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 10:55:18.343695 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application [None req-4e4c19b0-f7ad-4b19-a5df-a0913b4ae742 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 26 10:55:18.343695 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 10:55:18.343695 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 10:55:18.343695 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 10:55:18.343695 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 10:55:18.343695 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 10:55:18.343695 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 10:55:18.343695 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 10:55:18.343695 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 10:55:18.343695 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 10:55:18.343695 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 10:55:18.343695 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 10:55:18.343695 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 10:55:18.343695 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 10:55:18.343695 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 10:55:18.343695 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 10:55:18.344713 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 10:55:18.344713 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 10:55:18.344713 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 10:55:18.344713 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 10:55:18.344713 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 10:55:18.344713 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 10:55:18.344713 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 10:55:18.344713 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 10:55:18.344713 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 10:55:18.344713 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 10:55:18.344713 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 10:55:18.344713 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application with Lock( Aug 26 10:55:18.344713 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 10:55:18.344713 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._enter() Aug 26 10:55:18.344713 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 10:55:18.344713 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 10:55:18.345734 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 10:55:18.345734 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.creator() Aug 26 10:55:18.345734 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 10:55:18.345734 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application created_value = creator( Aug 26 10:55:18.345734 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 10:55:18.345734 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 10:55:18.345734 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 10:55:18.345734 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 10:55:18.345734 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 10:55:18.345734 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 10:55:18.345734 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 26 10:55:18.345734 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Aug 26 10:55:18.345734 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 622/1249] 10.222.0.40 () {56 vars in 1096 bytes} [Fri Aug 26 10:55:18 2022] GET /identity/v3/roles/admin => generated 83 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 10:55:18.350929 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e868578-e587-44a3-b44d-dd40b4670363 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:55:18.463972 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e868578-e587-44a3-b44d-dd40b4670363 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:55:18.500468 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:55:18.505909 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8de465e-73a3-4ad6-93e6-3dfc48c23e6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:55:18.506577 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8de465e-73a3-4ad6-93e6-3dfc48c23e6a None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:55:18.506855 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8de465e-73a3-4ad6-93e6-3dfc48c23e6a None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:55:18.507121 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8de465e-73a3-4ad6-93e6-3dfc48c23e6a None admin] PATH_INFO: `/v3/roles` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:55:18.507773 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8de465e-73a3-4ad6-93e6-3dfc48c23e6a None admin] RBAC: Adding query filter params (name=admin) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 10:55:18.508067 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8de465e-73a3-4ad6-93e6-3dfc48c23e6a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:55:18.509432 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8de465e-73a3-4ad6-93e6-3dfc48c23e6a None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:55:18.522693 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 628/1250] 10.222.0.40 () {56 vars in 1099 bytes} [Fri Aug 26 10:55:18 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 10:55:18.524194 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4c19b0-f7ad-4b19-a5df-a0913b4ae742 None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:55:18.615429 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:55:18.642241 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4c19b0-f7ad-4b19-a5df-a0913b4ae742 None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:55:18.679923 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92448015-411b-43da-adc8-04d441ed42d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:55:18.680599 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92448015-411b-43da-adc8-04d441ed42d1 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:55:18.680874 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92448015-411b-43da-adc8-04d441ed42d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:55:18.681144 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92448015-411b-43da-adc8-04d441ed42d1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:55:18.692377 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92448015-411b-43da-adc8-04d441ed42d1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:55:18.693829 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92448015-411b-43da-adc8-04d441ed42d1 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:55:18.710413 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application [None req-92448015-411b-43da-adc8-04d441ed42d1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 10:55:18.710413 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 10:55:18.710413 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 10:55:18.710413 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 10:55:18.710413 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 10:55:18.710413 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 10:55:18.710413 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 10:55:18.710413 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 10:55:18.710413 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 10:55:18.710413 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 10:55:18.710413 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 10:55:18.710413 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 10:55:18.710413 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 10:55:18.710413 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 10:55:18.710413 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 10:55:18.710413 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 10:55:18.711813 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 10:55:18.711813 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 10:55:18.711813 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 10:55:18.711813 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 10:55:18.711813 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 10:55:18.711813 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 10:55:18.711813 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 10:55:18.711813 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application with Lock( Aug 26 10:55:18.711813 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 10:55:18.711813 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._enter() Aug 26 10:55:18.711813 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 10:55:18.711813 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 10:55:18.711813 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 10:55:18.711813 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.creator() Aug 26 10:55:18.711813 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 10:55:18.711813 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application created_value = creator( Aug 26 10:55:18.711813 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 10:55:18.712960 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 10:55:18.712960 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 10:55:18.712960 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Aug 26 10:55:18.712960 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 623/1251] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Aug 26 10:55:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 10:55:18.717895 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8de465e-73a3-4ad6-93e6-3dfc48c23e6a None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:55:18.824726 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8de465e-73a3-4ad6-93e6-3dfc48c23e6a None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:55:18.859980 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5293145b-e94d-4708-8920-ba67ef309233 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:55:18.860863 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5293145b-e94d-4708-8920-ba67ef309233 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:55:18.861140 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5293145b-e94d-4708-8920-ba67ef309233 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:55:18.861412 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5293145b-e94d-4708-8920-ba67ef309233 None admin] PATH_INFO: `/v3/domains` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:55:18.862088 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5293145b-e94d-4708-8920-ba67ef309233 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 10:55:18.862383 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5293145b-e94d-4708-8920-ba67ef309233 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:55:18.863737 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5293145b-e94d-4708-8920-ba67ef309233 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:55:18.876352 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 629/1252] 10.222.0.40 () {56 vars in 1109 bytes} [Fri Aug 26 10:55:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 10:55:18.882278 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92448015-411b-43da-adc8-04d441ed42d1 None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:55:18.994382 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92448015-411b-43da-adc8-04d441ed42d1 None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:55:19.021535 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:55:19.032069 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902da0a6-4685-4061-84d8-730c4f2ddeeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:55:19.032772 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902da0a6-4685-4061-84d8-730c4f2ddeeb None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:55:19.033095 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902da0a6-4685-4061-84d8-730c4f2ddeeb None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:55:19.033319 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902da0a6-4685-4061-84d8-730c4f2ddeeb None admin] PATH_INFO: `/v3/users/placement` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:55:19.040987 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-902da0a6-4685-4061-84d8-730c4f2ddeeb None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:55:19.042454 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-902da0a6-4685-4061-84d8-730c4f2ddeeb None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:55:19.055000 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application [None req-902da0a6-4685-4061-84d8-730c4f2ddeeb None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 26 10:55:19.055000 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 10:55:19.055000 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 10:55:19.055000 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 10:55:19.055000 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 10:55:19.055000 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 10:55:19.055000 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 10:55:19.055000 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 10:55:19.055000 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 10:55:19.055000 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 10:55:19.055000 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 10:55:19.055000 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 10:55:19.055000 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 26 10:55:19.055000 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 26 10:55:19.055000 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 26 10:55:19.055000 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 26 10:55:19.056266 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 10:55:19.056266 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 10:55:19.056266 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 26 10:55:19.056266 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 10:55:19.056266 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 26 10:55:19.056266 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 10:55:19.056266 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 10:55:19.056266 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 10:55:19.056266 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 10:55:19.056266 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 10:55:19.056266 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 10:55:19.056266 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application with Lock( Aug 26 10:55:19.056266 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 10:55:19.056266 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._enter() Aug 26 10:55:19.056266 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 10:55:19.056266 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 10:55:19.056266 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 10:55:19.056266 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.creator() Aug 26 10:55:19.057237 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 10:55:19.057237 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application created_value = creator( Aug 26 10:55:19.057237 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 26 10:55:19.057237 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 26 10:55:19.057237 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 26 10:55:19.057237 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 26 10:55:19.057237 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 26 10:55:19.057237 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 26 10:55:19.057237 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 26 10:55:19.057237 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Aug 26 10:55:19.057237 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 624/1253] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Aug 26 10:55:18 2022] GET /identity/v3/users/placement => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 10:55:19.062583 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5293145b-e94d-4708-8920-ba67ef309233 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:55:19.126392 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:55:19.176473 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5293145b-e94d-4708-8920-ba67ef309233 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:55:19.214633 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85b45bb-3526-4bb9-87d5-1d192e92690c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:55:19.215237 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85b45bb-3526-4bb9-87d5-1d192e92690c None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:55:19.215516 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85b45bb-3526-4bb9-87d5-1d192e92690c None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:55:19.215786 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85b45bb-3526-4bb9-87d5-1d192e92690c None admin] PATH_INFO: `/v3/users` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:55:19.216573 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c85b45bb-3526-4bb9-87d5-1d192e92690c None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 10:55:19.217671 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c85b45bb-3526-4bb9-87d5-1d192e92690c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:55:19.218829 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c85b45bb-3526-4bb9-87d5-1d192e92690c None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:55:19.239107 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 630/1254] 10.222.0.40 () {56 vars in 1143 bytes} [Fri Aug 26 10:55:19 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 10:55:19.245914 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902da0a6-4685-4061-84d8-730c4f2ddeeb None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:55:19.378874 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902da0a6-4685-4061-84d8-730c4f2ddeeb None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:55:19.413839 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52169e4-7900-4290-a1f5-5f78f7d0093c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:55:19.414465 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52169e4-7900-4290-a1f5-5f78f7d0093c None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:55:19.414741 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52169e4-7900-4290-a1f5-5f78f7d0093c None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:55:19.415009 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52169e4-7900-4290-a1f5-5f78f7d0093c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:55:19.471022 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:55:19.484600 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f52169e4-7900-4290-a1f5-5f78f7d0093c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:55:19.486353 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f52169e4-7900-4290-a1f5-5f78f7d0093c None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:55:19.599247 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 625/1255] 10.222.0.40 () {58 vars in 1282 bytes} [Fri Aug 26 10:55:19 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 26 10:55:19.600577 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85b45bb-3526-4bb9-87d5-1d192e92690c None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:55:19.713407 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85b45bb-3526-4bb9-87d5-1d192e92690c None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:55:19.727277 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:55:19.752399 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c056029-7658-43be-8f5e-9220f0b8c079 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:55:19.752945 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c056029-7658-43be-8f5e-9220f0b8c079 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:55:19.753218 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c056029-7658-43be-8f5e-9220f0b8c079 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:55:19.753570 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c056029-7658-43be-8f5e-9220f0b8c079 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:55:19.765282 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c056029-7658-43be-8f5e-9220f0b8c079 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:55:19.767051 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c056029-7658-43be-8f5e-9220f0b8c079 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:55:19.782835 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application [None req-2c056029-7658-43be-8f5e-9220f0b8c079 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 10:55:19.782835 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 10:55:19.782835 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 10:55:19.782835 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 10:55:19.782835 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 10:55:19.782835 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 10:55:19.782835 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 10:55:19.782835 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 10:55:19.782835 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 10:55:19.782835 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 10:55:19.782835 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 10:55:19.782835 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 10:55:19.782835 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 10:55:19.782835 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 10:55:19.782835 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 10:55:19.782835 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 10:55:19.783996 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 10:55:19.783996 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 10:55:19.783996 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 10:55:19.783996 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 10:55:19.783996 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 10:55:19.783996 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 10:55:19.783996 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 10:55:19.783996 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application with Lock( Aug 26 10:55:19.783996 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 10:55:19.783996 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._enter() Aug 26 10:55:19.783996 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 10:55:19.783996 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 10:55:19.783996 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 10:55:19.783996 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.creator() Aug 26 10:55:19.783996 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 10:55:19.783996 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application created_value = creator( Aug 26 10:55:19.783996 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 10:55:19.785080 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 10:55:19.785080 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 10:55:19.785080 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Aug 26 10:55:19.785080 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 631/1256] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Aug 26 10:55:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 10:55:19.789944 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52169e4-7900-4290-a1f5-5f78f7d0093c None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:55:19.891183 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:55:19.904808 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52169e4-7900-4290-a1f5-5f78f7d0093c None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:55:19.946965 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7583099a-7b7c-4a15-9da5-94b72fbbc59d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:55:19.947632 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7583099a-7b7c-4a15-9da5-94b72fbbc59d None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:55:19.948047 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7583099a-7b7c-4a15-9da5-94b72fbbc59d None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:55:19.948335 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7583099a-7b7c-4a15-9da5-94b72fbbc59d None admin] PATH_INFO: `/v3/domains` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:55:19.949037 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7583099a-7b7c-4a15-9da5-94b72fbbc59d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 10:55:19.949341 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7583099a-7b7c-4a15-9da5-94b72fbbc59d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:55:19.951106 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7583099a-7b7c-4a15-9da5-94b72fbbc59d None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:55:19.965861 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 626/1257] 10.222.0.40 () {56 vars in 1109 bytes} [Fri Aug 26 10:55:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 10:55:19.971905 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c056029-7658-43be-8f5e-9220f0b8c079 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:55:20.086284 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c056029-7658-43be-8f5e-9220f0b8c079 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:55:20.123897 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acfb9df9-66fa-45f0-a56e-5a4986a88e22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:55:20.124551 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfb9df9-66fa-45f0-a56e-5a4986a88e22 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:55:20.124826 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfb9df9-66fa-45f0-a56e-5a4986a88e22 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:55:20.125090 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfb9df9-66fa-45f0-a56e-5a4986a88e22 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:55:20.132787 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acfb9df9-66fa-45f0-a56e-5a4986a88e22 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:55:20.134221 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acfb9df9-66fa-45f0-a56e-5a4986a88e22 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:55:20.147416 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application [None req-acfb9df9-66fa-45f0-a56e-5a4986a88e22 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 10:55:20.147416 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 10:55:20.147416 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 10:55:20.147416 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 10:55:20.147416 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 10:55:20.147416 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 10:55:20.147416 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 10:55:20.147416 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 10:55:20.147416 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 10:55:20.147416 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 10:55:20.147416 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 10:55:20.147416 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 10:55:20.147416 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 26 10:55:20.147416 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 26 10:55:20.147416 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 26 10:55:20.148580 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 26 10:55:20.148580 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 10:55:20.148580 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 10:55:20.148580 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 10:55:20.148580 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 10:55:20.148580 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 10:55:20.148580 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 10:55:20.148580 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 10:55:20.148580 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application with Lock( Aug 26 10:55:20.148580 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 10:55:20.148580 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._enter() Aug 26 10:55:20.148580 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 10:55:20.148580 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 10:55:20.148580 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 10:55:20.148580 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.creator() Aug 26 10:55:20.148580 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 10:55:20.148580 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application created_value = creator( Aug 26 10:55:20.149864 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 26 10:55:20.149864 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 26 10:55:20.149864 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 26 10:55:20.149864 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 26 10:55:20.149864 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 26 10:55:20.149864 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 26 10:55:20.149864 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 10:55:20.149864 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Aug 26 10:55:20.149864 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 632/1258] 10.222.0.40 () {56 vars in 1111 bytes} [Fri Aug 26 10:55:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 10:55:20.154794 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7583099a-7b7c-4a15-9da5-94b72fbbc59d None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:55:20.266619 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7583099a-7b7c-4a15-9da5-94b72fbbc59d None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:55:20.308229 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e71f769-22e7-4b52-9c8b-ea036ee4f355 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:55:20.308842 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e71f769-22e7-4b52-9c8b-ea036ee4f355 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:55:20.309120 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e71f769-22e7-4b52-9c8b-ea036ee4f355 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:55:20.309388 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e71f769-22e7-4b52-9c8b-ea036ee4f355 None admin] PATH_INFO: `/v3/projects` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:55:20.310121 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e71f769-22e7-4b52-9c8b-ea036ee4f355 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 10:55:20.310486 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e71f769-22e7-4b52-9c8b-ea036ee4f355 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:55:20.311930 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e71f769-22e7-4b52-9c8b-ea036ee4f355 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:55:20.325065 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 627/1259] 10.222.0.40 () {56 vars in 1148 bytes} [Fri Aug 26 10:55:20 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 10:55:20.331708 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acfb9df9-66fa-45f0-a56e-5a4986a88e22 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:55:20.359935 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:55:20.436130 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acfb9df9-66fa-45f0-a56e-5a4986a88e22 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:55:20.473215 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30241bd-3212-4d2e-b24b-9328072fbfdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:55:20.473818 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30241bd-3212-4d2e-b24b-9328072fbfdd None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:55:20.474117 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30241bd-3212-4d2e-b24b-9328072fbfdd None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:55:20.474389 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30241bd-3212-4d2e-b24b-9328072fbfdd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:55:20.475096 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d30241bd-3212-4d2e-b24b-9328072fbfdd None admin] RBAC: Adding query filter params (role.id=1f97e4602e2e4d6498913a81f9ae8d6a, scope.project.id=0e30ccf0c1204727b786876c141dbcd2, user.id=f29c5dfed31f42a9b5cbc0cd72e9fdc5) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 10:55:20.475394 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d30241bd-3212-4d2e-b24b-9328072fbfdd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:55:20.476805 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d30241bd-3212-4d2e-b24b-9328072fbfdd None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:55:20.482178 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 633/1260] 10.222.0.40 () {56 vars in 1374 bytes} [Fri Aug 26 10:55:20 2022] GET /identity/v3/role_assignments?user.id=f29c5dfed31f42a9b5cbc0cd72e9fdc5&scope.project.id=0e30ccf0c1204727b786876c141dbcd2&role.id=1f97e4602e2e4d6498913a81f9ae8d6a => generated 259 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 10:55:22.381170 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4171313-ebbd-432c-b617-d1dba2e7df61 None None] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:55:22.381675 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4171313-ebbd-432c-b617-d1dba2e7df61 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:55:22.381765 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4171313-ebbd-432c-b617-d1dba2e7df61 None None] PATH_INFO: `/` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:55:22.383085 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 628/1261] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 26 10:55:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 10:55:22.388208 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa289e1a-33a5-430a-b0eb-5c8074a0a221 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:55:22.388619 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa289e1a-33a5-430a-b0eb-5c8074a0a221 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:55:22.389086 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa289e1a-33a5-430a-b0eb-5c8074a0a221 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:55:22.406655 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-aa289e1a-33a5-430a-b0eb-5c8074a0a221 None None] MFA Rules not processed for user `ca155a48e2024c32af3fd2e19e5fac11`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 10:55:22.407831 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-aa289e1a-33a5-430a-b0eb-5c8074a0a221 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 10:55:22.472689 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 634/1262] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 26 10:55:22 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 26 10:55:22.746836 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4171313-ebbd-432c-b617-d1dba2e7df61 None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:55:22.757504 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:55:22.847350 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4171313-ebbd-432c-b617-d1dba2e7df61 None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:55:22.880493 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5677e361-dd11-4133-94f2-ae55e0eaa672 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:55:22.881110 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5677e361-dd11-4133-94f2-ae55e0eaa672 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:55:22.881383 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5677e361-dd11-4133-94f2-ae55e0eaa672 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:55:22.881646 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5677e361-dd11-4133-94f2-ae55e0eaa672 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:55:22.891718 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5677e361-dd11-4133-94f2-ae55e0eaa672 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 10:55:22.896348 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application [None req-5677e361-dd11-4133-94f2-ae55e0eaa672 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 26 10:55:22.896348 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 10:55:22.896348 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 10:55:22.896348 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 10:55:22.896348 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 10:55:22.896348 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 10:55:22.896348 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 10:55:22.896348 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 10:55:22.896348 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 10:55:22.896348 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 10:55:22.896348 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 10:55:22.896348 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 10:55:22.896348 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 10:55:22.896348 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 10:55:22.896348 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 10:55:22.896348 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 10:55:22.898136 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 10:55:22.898136 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 10:55:22.898136 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 10:55:22.898136 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 10:55:22.898136 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 10:55:22.898136 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 10:55:22.898136 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 10:55:22.898136 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 10:55:22.898136 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 10:55:22.898136 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 10:55:22.898136 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 10:55:22.898136 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application with Lock( Aug 26 10:55:22.898136 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 10:55:22.898136 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._enter() Aug 26 10:55:22.898136 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 10:55:22.898136 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 10:55:22.899318 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 10:55:22.899318 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.creator() Aug 26 10:55:22.899318 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 10:55:22.899318 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application created_value = creator( Aug 26 10:55:22.899318 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 10:55:22.899318 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 10:55:22.899318 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 10:55:22.899318 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 10:55:22.899318 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 10:55:22.899318 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 10:55:22.899318 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 26 10:55:22.899318 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Aug 26 10:55:22.899318 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 629/1263] 10.222.0.40 () {56 vars in 1096 bytes} [Fri Aug 26 10:55:22 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 10:55:22.903405 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa289e1a-33a5-430a-b0eb-5c8074a0a221 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:55:22.922303 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:55:23.001170 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa289e1a-33a5-430a-b0eb-5c8074a0a221 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:55:23.033872 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b431db-e4e8-4776-91a5-634e1984315c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:55:23.034486 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b431db-e4e8-4776-91a5-634e1984315c None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:55:23.034760 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b431db-e4e8-4776-91a5-634e1984315c None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:55:23.035029 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b431db-e4e8-4776-91a5-634e1984315c None admin] PATH_INFO: `/v3/roles` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:55:23.035675 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0b431db-e4e8-4776-91a5-634e1984315c None admin] RBAC: Adding query filter params (name=admin) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 10:55:23.035971 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0b431db-e4e8-4776-91a5-634e1984315c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:55:23.037316 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0b431db-e4e8-4776-91a5-634e1984315c None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:55:23.045062 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 635/1264] 10.222.0.40 () {56 vars in 1099 bytes} [Fri Aug 26 10:55:22 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 10:55:23.050737 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5677e361-dd11-4133-94f2-ae55e0eaa672 None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:55:23.116316 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:55:23.148499 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5677e361-dd11-4133-94f2-ae55e0eaa672 None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:55:23.181119 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408da543-ff6a-41e1-926e-02db74bb6bb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:55:23.181730 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408da543-ff6a-41e1-926e-02db74bb6bb9 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:55:23.182026 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408da543-ff6a-41e1-926e-02db74bb6bb9 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:55:23.182294 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408da543-ff6a-41e1-926e-02db74bb6bb9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:55:23.192210 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-408da543-ff6a-41e1-926e-02db74bb6bb9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:55:23.193624 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-408da543-ff6a-41e1-926e-02db74bb6bb9 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:55:23.208342 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application [None req-408da543-ff6a-41e1-926e-02db74bb6bb9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 10:55:23.208342 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 10:55:23.208342 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 10:55:23.208342 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 10:55:23.208342 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 10:55:23.208342 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 10:55:23.208342 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 10:55:23.208342 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 10:55:23.208342 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 10:55:23.208342 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 10:55:23.208342 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 10:55:23.208342 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 10:55:23.208342 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 10:55:23.208342 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 10:55:23.208342 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 10:55:23.208342 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 10:55:23.209493 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 10:55:23.209493 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 10:55:23.209493 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 10:55:23.209493 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 10:55:23.209493 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 10:55:23.209493 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 10:55:23.209493 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 10:55:23.209493 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application with Lock( Aug 26 10:55:23.209493 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 10:55:23.209493 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._enter() Aug 26 10:55:23.209493 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 10:55:23.209493 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 10:55:23.209493 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 10:55:23.209493 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.creator() Aug 26 10:55:23.209493 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 10:55:23.209493 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application created_value = creator( Aug 26 10:55:23.209493 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 10:55:23.210653 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 10:55:23.210653 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 10:55:23.210653 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Aug 26 10:55:23.210653 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 630/1265] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Aug 26 10:55:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 10:55:23.215432 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b431db-e4e8-4776-91a5-634e1984315c None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:55:23.309504 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:55:23.330656 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b431db-e4e8-4776-91a5-634e1984315c None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:55:23.362938 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea6334d-3158-473d-81db-a552366c0320 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:55:23.363538 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea6334d-3158-473d-81db-a552366c0320 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:55:23.363821 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea6334d-3158-473d-81db-a552366c0320 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:55:23.364091 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea6334d-3158-473d-81db-a552366c0320 None admin] PATH_INFO: `/v3/domains` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:55:23.364725 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea6334d-3158-473d-81db-a552366c0320 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 10:55:23.365027 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea6334d-3158-473d-81db-a552366c0320 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:55:23.366421 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea6334d-3158-473d-81db-a552366c0320 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:55:23.378317 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 636/1266] 10.222.0.40 () {56 vars in 1109 bytes} [Fri Aug 26 10:55:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 10:55:23.384112 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408da543-ff6a-41e1-926e-02db74bb6bb9 None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:55:23.480551 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408da543-ff6a-41e1-926e-02db74bb6bb9 None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:55:23.499951 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:55:23.514535 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b20b623-1222-4829-9bed-5b8866e38cbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:55:23.515148 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b20b623-1222-4829-9bed-5b8866e38cbf None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:55:23.515420 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b20b623-1222-4829-9bed-5b8866e38cbf None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:55:23.515686 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b20b623-1222-4829-9bed-5b8866e38cbf None admin] PATH_INFO: `/v3/users/placement` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:55:23.522832 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b20b623-1222-4829-9bed-5b8866e38cbf None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:55:23.524312 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b20b623-1222-4829-9bed-5b8866e38cbf None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:55:23.535102 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application [None req-3b20b623-1222-4829-9bed-5b8866e38cbf None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 26 10:55:23.535102 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 10:55:23.535102 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 10:55:23.535102 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 10:55:23.535102 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 10:55:23.535102 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 10:55:23.535102 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 10:55:23.535102 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 10:55:23.535102 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 10:55:23.535102 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 10:55:23.535102 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 10:55:23.535102 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 10:55:23.535102 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 26 10:55:23.535102 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 26 10:55:23.535102 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 26 10:55:23.535102 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 26 10:55:23.536234 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 10:55:23.536234 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 10:55:23.536234 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 26 10:55:23.536234 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 10:55:23.536234 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 26 10:55:23.536234 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 10:55:23.536234 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 10:55:23.536234 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 10:55:23.536234 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 10:55:23.536234 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 10:55:23.536234 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 10:55:23.536234 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application with Lock( Aug 26 10:55:23.536234 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 10:55:23.536234 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._enter() Aug 26 10:55:23.536234 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 10:55:23.536234 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 10:55:23.536234 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 10:55:23.536234 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.creator() Aug 26 10:55:23.537425 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 10:55:23.537425 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application created_value = creator( Aug 26 10:55:23.537425 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 26 10:55:23.537425 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 26 10:55:23.537425 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 26 10:55:23.537425 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 26 10:55:23.537425 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 26 10:55:23.537425 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 26 10:55:23.537425 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 26 10:55:23.537425 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Aug 26 10:55:23.537425 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 631/1267] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Aug 26 10:55:23 2022] GET /identity/v3/users/placement => generated 87 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 10:55:23.542098 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea6334d-3158-473d-81db-a552366c0320 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:55:23.639139 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea6334d-3158-473d-81db-a552366c0320 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:55:23.671893 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca615e9-392f-41d5-afcf-64b0f121f50b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:55:23.672474 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca615e9-392f-41d5-afcf-64b0f121f50b None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:55:23.672748 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca615e9-392f-41d5-afcf-64b0f121f50b None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:55:23.673016 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca615e9-392f-41d5-afcf-64b0f121f50b None admin] PATH_INFO: `/v3/users` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:55:23.673726 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca615e9-392f-41d5-afcf-64b0f121f50b None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 10:55:23.674050 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca615e9-392f-41d5-afcf-64b0f121f50b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:55:23.675466 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca615e9-392f-41d5-afcf-64b0f121f50b None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:55:23.683756 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:55:23.694707 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 637/1268] 10.222.0.40 () {56 vars in 1143 bytes} [Fri Aug 26 10:55:23 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 10:55:23.702061 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b20b623-1222-4829-9bed-5b8866e38cbf None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:55:23.800681 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b20b623-1222-4829-9bed-5b8866e38cbf None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:55:23.833298 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32caa18-82f4-4f28-9e82-c825fc01d10d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:55:23.833904 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32caa18-82f4-4f28-9e82-c825fc01d10d None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:55:23.834208 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32caa18-82f4-4f28-9e82-c825fc01d10d None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:55:23.834476 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32caa18-82f4-4f28-9e82-c825fc01d10d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:55:23.845118 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e32caa18-82f4-4f28-9e82-c825fc01d10d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:55:23.846570 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e32caa18-82f4-4f28-9e82-c825fc01d10d None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:55:23.860859 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application [None req-e32caa18-82f4-4f28-9e82-c825fc01d10d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 10:55:23.860859 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 10:55:23.860859 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 10:55:23.860859 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 10:55:23.860859 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 10:55:23.860859 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 10:55:23.860859 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 10:55:23.860859 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 10:55:23.860859 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 10:55:23.860859 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 10:55:23.860859 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 10:55:23.860859 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 10:55:23.860859 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 10:55:23.860859 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 10:55:23.860859 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 10:55:23.860859 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 10:55:23.862052 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 10:55:23.862052 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 10:55:23.862052 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 10:55:23.862052 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 10:55:23.862052 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 10:55:23.862052 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 10:55:23.862052 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 10:55:23.862052 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application with Lock( Aug 26 10:55:23.862052 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 10:55:23.862052 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._enter() Aug 26 10:55:23.862052 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 10:55:23.862052 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 10:55:23.862052 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 10:55:23.862052 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.creator() Aug 26 10:55:23.862052 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 10:55:23.862052 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application created_value = creator( Aug 26 10:55:23.862052 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 10:55:23.863120 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 10:55:23.863120 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 10:55:23.863120 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Aug 26 10:55:23.863120 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 632/1269] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Aug 26 10:55:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 10:55:23.867743 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca615e9-392f-41d5-afcf-64b0f121f50b None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:55:23.964476 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca615e9-392f-41d5-afcf-64b0f121f50b None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:55:23.997475 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0540a564-3839-4921-aa42-bb6ebe0e66a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:55:23.998365 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0540a564-3839-4921-aa42-bb6ebe0e66a9 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:55:23.998656 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0540a564-3839-4921-aa42-bb6ebe0e66a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:55:23.998925 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0540a564-3839-4921-aa42-bb6ebe0e66a9 None admin] PATH_INFO: `/v3/domains` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:55:23.999561 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0540a564-3839-4921-aa42-bb6ebe0e66a9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 10:55:23.999856 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0540a564-3839-4921-aa42-bb6ebe0e66a9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:55:24.001207 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0540a564-3839-4921-aa42-bb6ebe0e66a9 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:55:24.011918 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 638/1270] 10.222.0.40 () {56 vars in 1109 bytes} [Fri Aug 26 10:55:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 10:55:24.018407 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32caa18-82f4-4f28-9e82-c825fc01d10d None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:55:24.049679 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:55:24.116278 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32caa18-82f4-4f28-9e82-c825fc01d10d None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:55:24.148546 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0abcde85-63a2-42c1-a08d-21772b3f6ea2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:55:24.149151 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abcde85-63a2-42c1-a08d-21772b3f6ea2 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:55:24.149446 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abcde85-63a2-42c1-a08d-21772b3f6ea2 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:55:24.149710 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abcde85-63a2-42c1-a08d-21772b3f6ea2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:55:24.156735 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0abcde85-63a2-42c1-a08d-21772b3f6ea2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:55:24.158260 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0abcde85-63a2-42c1-a08d-21772b3f6ea2 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:55:24.169114 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application [None req-0abcde85-63a2-42c1-a08d-21772b3f6ea2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 10:55:24.169114 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 10:55:24.169114 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 10:55:24.169114 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 10:55:24.169114 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 10:55:24.169114 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 10:55:24.169114 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 10:55:24.169114 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 10:55:24.169114 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 10:55:24.169114 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 10:55:24.169114 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 10:55:24.169114 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 10:55:24.169114 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 26 10:55:24.169114 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 26 10:55:24.169114 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 26 10:55:24.170743 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 26 10:55:24.170743 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 10:55:24.170743 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 10:55:24.170743 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 10:55:24.170743 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 10:55:24.170743 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 10:55:24.170743 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 10:55:24.170743 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 10:55:24.170743 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application with Lock( Aug 26 10:55:24.170743 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 10:55:24.170743 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._enter() Aug 26 10:55:24.170743 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 10:55:24.170743 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 10:55:24.170743 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 10:55:24.170743 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.creator() Aug 26 10:55:24.170743 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 10:55:24.170743 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application created_value = creator( Aug 26 10:55:24.171947 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 26 10:55:24.171947 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 26 10:55:24.171947 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 26 10:55:24.171947 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 26 10:55:24.171947 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 26 10:55:24.171947 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 26 10:55:24.171947 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 10:55:24.171947 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Aug 26 10:55:24.171947 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 633/1271] 10.222.0.40 () {56 vars in 1111 bytes} [Fri Aug 26 10:55:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 10:55:24.176631 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0540a564-3839-4921-aa42-bb6ebe0e66a9 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:55:24.231846 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:55:24.277575 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0540a564-3839-4921-aa42-bb6ebe0e66a9 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:55:24.311785 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633331b5-76ed-4ab7-bb53-ed9a19f7e2ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:55:24.312400 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633331b5-76ed-4ab7-bb53-ed9a19f7e2ad None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:55:24.312676 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633331b5-76ed-4ab7-bb53-ed9a19f7e2ad None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:55:24.312942 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633331b5-76ed-4ab7-bb53-ed9a19f7e2ad None admin] PATH_INFO: `/v3/projects` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:55:24.313619 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-633331b5-76ed-4ab7-bb53-ed9a19f7e2ad None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 10:55:24.313916 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-633331b5-76ed-4ab7-bb53-ed9a19f7e2ad None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:55:24.315357 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-633331b5-76ed-4ab7-bb53-ed9a19f7e2ad None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:55:24.326812 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 639/1272] 10.222.0.40 () {56 vars in 1148 bytes} [Fri Aug 26 10:55:24 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 10:55:24.332794 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0abcde85-63a2-42c1-a08d-21772b3f6ea2 None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:55:24.423206 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:55:24.432051 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0abcde85-63a2-42c1-a08d-21772b3f6ea2 None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:55:24.465054 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37467844-9c98-47b3-bdbf-184c005352f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:55:24.465700 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37467844-9c98-47b3-bdbf-184c005352f7 None admin] REQUEST_METHOD: `PUT` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:55:24.465997 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37467844-9c98-47b3-bdbf-184c005352f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:55:24.466275 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37467844-9c98-47b3-bdbf-184c005352f7 None admin] PATH_INFO: `/v3/projects/0e30ccf0c1204727b786876c141dbcd2/users/f29c5dfed31f42a9b5cbc0cd72e9fdc5/roles/1f97e4602e2e4d6498913a81f9ae8d6a` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:55:24.468774 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37467844-9c98-47b3-bdbf-184c005352f7 None admin] RBAC: Authorizing `identity:create_grant(project_id=0e30ccf0c1204727b786876c141dbcd2, user_id=f29c5dfed31f42a9b5cbc0cd72e9fdc5, role_id=1f97e4602e2e4d6498913a81f9ae8d6a)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:55:24.470225 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37467844-9c98-47b3-bdbf-184c005352f7 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:55:24.482616 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 634/1273] 10.222.0.40 () {58 vars in 1439 bytes} [Fri Aug 26 10:55:24 2022] PUT /identity/v3/projects/0e30ccf0c1204727b786876c141dbcd2/users/f29c5dfed31f42a9b5cbc0cd72e9fdc5/roles/1f97e4602e2e4d6498913a81f9ae8d6a => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 10:55:26.335524 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a1950e-9a00-469d-ab2a-260e91fdfc97 None None] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:55:26.336018 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a1950e-9a00-469d-ab2a-260e91fdfc97 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:55:26.336018 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a1950e-9a00-469d-ab2a-260e91fdfc97 None None] PATH_INFO: `/` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:55:26.337319 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 640/1274] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 26 10:55:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 10:55:26.343290 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44087c28-80b4-4d78-b473-721ed107a9bd None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:55:26.343707 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44087c28-80b4-4d78-b473-721ed107a9bd None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:55:26.343815 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44087c28-80b4-4d78-b473-721ed107a9bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:55:26.362431 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-44087c28-80b4-4d78-b473-721ed107a9bd None None] MFA Rules not processed for user `ca155a48e2024c32af3fd2e19e5fac11`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 10:55:26.363615 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-44087c28-80b4-4d78-b473-721ed107a9bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 10:55:26.427917 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 635/1275] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 26 10:55:26 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 26 10:55:26.697153 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a1950e-9a00-469d-ab2a-260e91fdfc97 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:55:26.796943 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a1950e-9a00-469d-ab2a-260e91fdfc97 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:55:26.813461 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:55:26.830514 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff028f5-f241-4382-9f7c-ade27027831e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:55:26.831129 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff028f5-f241-4382-9f7c-ade27027831e None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:55:26.831404 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff028f5-f241-4382-9f7c-ade27027831e None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:55:26.831672 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff028f5-f241-4382-9f7c-ade27027831e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:55:26.841433 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff028f5-f241-4382-9f7c-ade27027831e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 10:55:26.846703 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application [None req-fff028f5-f241-4382-9f7c-ade27027831e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 26 10:55:26.846703 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 10:55:26.846703 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 10:55:26.846703 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 10:55:26.846703 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 10:55:26.846703 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 10:55:26.846703 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 10:55:26.846703 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 10:55:26.846703 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 10:55:26.846703 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 10:55:26.846703 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 10:55:26.846703 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 10:55:26.846703 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 10:55:26.846703 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 10:55:26.846703 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 10:55:26.846703 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 10:55:26.848696 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 10:55:26.848696 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 10:55:26.848696 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 10:55:26.848696 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 10:55:26.848696 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 10:55:26.848696 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 10:55:26.848696 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 10:55:26.848696 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 10:55:26.848696 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 10:55:26.848696 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 10:55:26.848696 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 10:55:26.848696 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application with Lock( Aug 26 10:55:26.848696 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 10:55:26.848696 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._enter() Aug 26 10:55:26.848696 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 10:55:26.848696 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 10:55:26.849872 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 10:55:26.849872 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.creator() Aug 26 10:55:26.849872 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 10:55:26.849872 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application created_value = creator( Aug 26 10:55:26.849872 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 10:55:26.849872 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 10:55:26.849872 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 10:55:26.849872 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 10:55:26.849872 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 10:55:26.849872 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 10:55:26.849872 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 26 10:55:26.849872 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Aug 26 10:55:26.849872 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 641/1276] 10.222.0.40 () {56 vars in 1096 bytes} [Fri Aug 26 10:55:26 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 10:55:26.854841 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44087c28-80b4-4d78-b473-721ed107a9bd None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:55:26.940264 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:55:26.959562 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44087c28-80b4-4d78-b473-721ed107a9bd None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:55:26.992810 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06553a78-6902-48a4-9a23-cd25da1f24a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:55:26.993407 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06553a78-6902-48a4-9a23-cd25da1f24a6 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:55:26.993689 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06553a78-6902-48a4-9a23-cd25da1f24a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:55:26.993962 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06553a78-6902-48a4-9a23-cd25da1f24a6 None admin] PATH_INFO: `/v3/roles` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:55:26.994649 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06553a78-6902-48a4-9a23-cd25da1f24a6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 10:55:26.994950 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06553a78-6902-48a4-9a23-cd25da1f24a6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:55:26.996304 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06553a78-6902-48a4-9a23-cd25da1f24a6 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:55:27.004327 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 636/1277] 10.222.0.40 () {56 vars in 1099 bytes} [Fri Aug 26 10:55:26 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 10:55:27.010015 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff028f5-f241-4382-9f7c-ade27027831e None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:55:27.108273 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff028f5-f241-4382-9f7c-ade27027831e None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:55:27.140550 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c494bd19-964a-420b-8287-187d4c470c95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:55:27.141172 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c494bd19-964a-420b-8287-187d4c470c95 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:55:27.141427 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c494bd19-964a-420b-8287-187d4c470c95 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:55:27.141691 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c494bd19-964a-420b-8287-187d4c470c95 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:55:27.151219 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c494bd19-964a-420b-8287-187d4c470c95 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:55:27.152654 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c494bd19-964a-420b-8287-187d4c470c95 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:55:27.167498 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application [None req-c494bd19-964a-420b-8287-187d4c470c95 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 10:55:27.167498 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 10:55:27.167498 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 10:55:27.167498 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 10:55:27.167498 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 10:55:27.167498 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 10:55:27.167498 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 10:55:27.167498 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 10:55:27.167498 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 10:55:27.167498 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 10:55:27.167498 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 10:55:27.167498 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 10:55:27.167498 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 10:55:27.167498 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 10:55:27.167498 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 10:55:27.167498 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 10:55:27.168612 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 10:55:27.168612 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 10:55:27.168612 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 10:55:27.168612 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 10:55:27.168612 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 10:55:27.168612 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 10:55:27.168612 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 10:55:27.168612 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application with Lock( Aug 26 10:55:27.168612 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 10:55:27.168612 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._enter() Aug 26 10:55:27.168612 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 10:55:27.168612 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 10:55:27.168612 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 10:55:27.168612 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.creator() Aug 26 10:55:27.168612 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 10:55:27.168612 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application created_value = creator( Aug 26 10:55:27.168612 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 10:55:27.170527 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 10:55:27.170527 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 10:55:27.170527 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Aug 26 10:55:27.170527 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 642/1278] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Aug 26 10:55:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 10:55:27.175366 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06553a78-6902-48a4-9a23-cd25da1f24a6 None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:55:27.276778 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06553a78-6902-48a4-9a23-cd25da1f24a6 None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:55:27.314442 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560c4753-8080-4f72-aa89-a53b260696f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:55:27.315048 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560c4753-8080-4f72-aa89-a53b260696f9 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:55:27.315344 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560c4753-8080-4f72-aa89-a53b260696f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:55:27.315623 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560c4753-8080-4f72-aa89-a53b260696f9 None admin] PATH_INFO: `/v3/domains` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:55:27.316280 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-560c4753-8080-4f72-aa89-a53b260696f9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 10:55:27.316584 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-560c4753-8080-4f72-aa89-a53b260696f9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:55:27.317943 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-560c4753-8080-4f72-aa89-a53b260696f9 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:55:27.329504 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 637/1279] 10.222.0.40 () {56 vars in 1109 bytes} [Fri Aug 26 10:55:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 10:55:27.335198 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c494bd19-964a-420b-8287-187d4c470c95 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:55:27.367360 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:55:27.435463 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c494bd19-964a-420b-8287-187d4c470c95 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:55:27.468294 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e0f464-ae6b-4b10-870c-d98ae0c7f554 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:55:27.468893 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e0f464-ae6b-4b10-870c-d98ae0c7f554 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:55:27.469168 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e0f464-ae6b-4b10-870c-d98ae0c7f554 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:55:27.469434 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e0f464-ae6b-4b10-870c-d98ae0c7f554 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:55:27.476456 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e0f464-ae6b-4b10-870c-d98ae0c7f554 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:55:27.477888 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e0f464-ae6b-4b10-870c-d98ae0c7f554 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:55:27.488898 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application [None req-f0e0f464-ae6b-4b10-870c-d98ae0c7f554 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 26 10:55:27.488898 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 10:55:27.488898 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 10:55:27.488898 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 10:55:27.488898 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 10:55:27.488898 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 10:55:27.488898 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 10:55:27.488898 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 10:55:27.488898 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 10:55:27.488898 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 10:55:27.488898 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 10:55:27.488898 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 10:55:27.488898 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 26 10:55:27.488898 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 26 10:55:27.488898 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 26 10:55:27.488898 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 26 10:55:27.490032 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 10:55:27.490032 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 10:55:27.490032 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 26 10:55:27.490032 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 10:55:27.490032 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 26 10:55:27.490032 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 10:55:27.490032 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 10:55:27.490032 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 10:55:27.490032 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 10:55:27.490032 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 10:55:27.490032 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 10:55:27.490032 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application with Lock( Aug 26 10:55:27.490032 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 10:55:27.490032 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._enter() Aug 26 10:55:27.490032 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 10:55:27.490032 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 10:55:27.490032 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 10:55:27.490032 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.creator() Aug 26 10:55:27.491011 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 10:55:27.491011 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application created_value = creator( Aug 26 10:55:27.491011 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 26 10:55:27.491011 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 26 10:55:27.491011 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 26 10:55:27.491011 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 26 10:55:27.491011 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 26 10:55:27.491011 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 26 10:55:27.491011 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 26 10:55:27.491011 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Aug 26 10:55:27.491011 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 643/1280] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Aug 26 10:55:27 2022] GET /identity/v3/users/placement => generated 87 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 10:55:27.495856 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560c4753-8080-4f72-aa89-a53b260696f9 None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:55:27.499684 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:55:27.593819 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560c4753-8080-4f72-aa89-a53b260696f9 None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:55:27.626796 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b83bd7-4651-495f-a58e-90557911ae88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:55:27.627392 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b83bd7-4651-495f-a58e-90557911ae88 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:55:27.627670 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b83bd7-4651-495f-a58e-90557911ae88 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:55:27.627940 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b83bd7-4651-495f-a58e-90557911ae88 None admin] PATH_INFO: `/v3/users` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:55:27.628659 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b83bd7-4651-495f-a58e-90557911ae88 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 10:55:27.628961 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b83bd7-4651-495f-a58e-90557911ae88 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:55:27.630405 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b83bd7-4651-495f-a58e-90557911ae88 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:55:27.647562 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 638/1281] 10.222.0.40 () {56 vars in 1143 bytes} [Fri Aug 26 10:55:27 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 10:55:27.653305 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e0f464-ae6b-4b10-870c-d98ae0c7f554 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:55:27.741567 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:55:27.751752 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e0f464-ae6b-4b10-870c-d98ae0c7f554 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:55:27.784669 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e4b2b0-a525-4009-bf0f-5e543b1a4ebc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:55:27.785262 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e4b2b0-a525-4009-bf0f-5e543b1a4ebc None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:55:27.785538 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e4b2b0-a525-4009-bf0f-5e543b1a4ebc None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:55:27.785807 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e4b2b0-a525-4009-bf0f-5e543b1a4ebc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:55:27.849728 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e4b2b0-a525-4009-bf0f-5e543b1a4ebc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:55:27.851122 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e4b2b0-a525-4009-bf0f-5e543b1a4ebc None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:55:27.936794 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:55:27.947066 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 644/1282] 10.222.0.40 () {58 vars in 1282 bytes} [Fri Aug 26 10:55:27 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 26 10:55:27.953513 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b83bd7-4651-495f-a58e-90557911ae88 None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:55:28.013286 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:55:28.051805 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b83bd7-4651-495f-a58e-90557911ae88 None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:55:28.085694 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd414dcc-d537-43c6-82e6-bafad6341948 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:55:28.086337 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd414dcc-d537-43c6-82e6-bafad6341948 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:55:28.086617 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd414dcc-d537-43c6-82e6-bafad6341948 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:55:28.086891 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd414dcc-d537-43c6-82e6-bafad6341948 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:55:28.097206 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd414dcc-d537-43c6-82e6-bafad6341948 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:55:28.098672 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd414dcc-d537-43c6-82e6-bafad6341948 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:55:28.113726 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application [None req-fd414dcc-d537-43c6-82e6-bafad6341948 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 10:55:28.113726 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 10:55:28.113726 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 10:55:28.113726 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 10:55:28.113726 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 10:55:28.113726 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 10:55:28.113726 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 10:55:28.113726 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 10:55:28.113726 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 10:55:28.113726 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 10:55:28.113726 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 10:55:28.113726 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 10:55:28.113726 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 10:55:28.113726 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 10:55:28.113726 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 10:55:28.113726 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 10:55:28.114843 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 10:55:28.114843 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 10:55:28.114843 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 10:55:28.114843 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 10:55:28.114843 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 10:55:28.114843 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 10:55:28.114843 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 10:55:28.114843 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application with Lock( Aug 26 10:55:28.114843 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 10:55:28.114843 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._enter() Aug 26 10:55:28.114843 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 10:55:28.114843 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 10:55:28.114843 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 10:55:28.114843 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.creator() Aug 26 10:55:28.114843 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 10:55:28.114843 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application created_value = creator( Aug 26 10:55:28.114843 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 10:55:28.115774 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 10:55:28.115774 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 10:55:28.115774 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Aug 26 10:55:28.118116 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 639/1283] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Aug 26 10:55:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 10:55:28.120370 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e4b2b0-a525-4009-bf0f-5e543b1a4ebc None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:55:28.214816 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e4b2b0-a525-4009-bf0f-5e543b1a4ebc None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:55:28.252953 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f914a7-0c67-43f8-965b-5e431c22a53e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:55:28.253944 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f914a7-0c67-43f8-965b-5e431c22a53e None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:55:28.255019 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f914a7-0c67-43f8-965b-5e431c22a53e None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:55:28.255333 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f914a7-0c67-43f8-965b-5e431c22a53e None admin] PATH_INFO: `/v3/domains` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:55:28.256572 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f914a7-0c67-43f8-965b-5e431c22a53e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 10:55:28.256913 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f914a7-0c67-43f8-965b-5e431c22a53e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:55:28.259049 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f914a7-0c67-43f8-965b-5e431c22a53e None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:55:28.271416 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 645/1284] 10.222.0.40 () {56 vars in 1109 bytes} [Fri Aug 26 10:55:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 10:55:28.277053 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd414dcc-d537-43c6-82e6-bafad6341948 None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:55:28.376433 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd414dcc-d537-43c6-82e6-bafad6341948 None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:55:28.389970 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:55:28.409834 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f29acf6-9a44-41d2-a96e-a9ab523da5c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:55:28.410484 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f29acf6-9a44-41d2-a96e-a9ab523da5c9 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:55:28.410763 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f29acf6-9a44-41d2-a96e-a9ab523da5c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:55:28.411034 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f29acf6-9a44-41d2-a96e-a9ab523da5c9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:55:28.417485 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f29acf6-9a44-41d2-a96e-a9ab523da5c9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:55:28.418948 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f29acf6-9a44-41d2-a96e-a9ab523da5c9 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:55:28.430269 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application [None req-3f29acf6-9a44-41d2-a96e-a9ab523da5c9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 10:55:28.430269 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 10:55:28.430269 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 10:55:28.430269 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 10:55:28.430269 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 10:55:28.430269 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 10:55:28.430269 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 10:55:28.430269 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 10:55:28.430269 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 10:55:28.430269 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 10:55:28.430269 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 10:55:28.430269 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 10:55:28.430269 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 26 10:55:28.430269 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 26 10:55:28.430269 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 26 10:55:28.431296 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 26 10:55:28.431296 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 10:55:28.431296 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 10:55:28.431296 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 10:55:28.431296 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 10:55:28.431296 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 10:55:28.431296 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 10:55:28.431296 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 10:55:28.431296 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application with Lock( Aug 26 10:55:28.431296 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 10:55:28.431296 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._enter() Aug 26 10:55:28.431296 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 10:55:28.431296 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 10:55:28.431296 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 10:55:28.431296 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.creator() Aug 26 10:55:28.431296 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 10:55:28.431296 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application created_value = creator( Aug 26 10:55:28.432365 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 26 10:55:28.432365 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 26 10:55:28.432365 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 26 10:55:28.432365 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 26 10:55:28.432365 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 26 10:55:28.432365 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 26 10:55:28.432365 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 10:55:28.432365 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Aug 26 10:55:28.432365 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 640/1285] 10.222.0.40 () {56 vars in 1111 bytes} [Fri Aug 26 10:55:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 10:55:28.437851 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f914a7-0c67-43f8-965b-5e431c22a53e None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:55:28.486571 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:55:28.538593 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f914a7-0c67-43f8-965b-5e431c22a53e None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:55:28.571071 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f132f6-a94e-43b3-b2c8-0d38d440cff9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:55:28.571665 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f132f6-a94e-43b3-b2c8-0d38d440cff9 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:55:28.571941 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f132f6-a94e-43b3-b2c8-0d38d440cff9 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:55:28.572210 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f132f6-a94e-43b3-b2c8-0d38d440cff9 None admin] PATH_INFO: `/v3/projects` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:55:28.572895 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f132f6-a94e-43b3-b2c8-0d38d440cff9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 10:55:28.573193 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f132f6-a94e-43b3-b2c8-0d38d440cff9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:55:28.574660 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f132f6-a94e-43b3-b2c8-0d38d440cff9 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:55:28.586641 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 646/1286] 10.222.0.40 () {56 vars in 1148 bytes} [Fri Aug 26 10:55:28 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 10:55:28.593270 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f29acf6-9a44-41d2-a96e-a9ab523da5c9 None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:55:28.690980 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f29acf6-9a44-41d2-a96e-a9ab523da5c9 None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:55:28.723788 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d8ab39-bc6f-4bc1-9dca-ecdcca04b89d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:55:28.724372 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d8ab39-bc6f-4bc1-9dca-ecdcca04b89d None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:55:28.724647 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d8ab39-bc6f-4bc1-9dca-ecdcca04b89d None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:55:28.724918 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d8ab39-bc6f-4bc1-9dca-ecdcca04b89d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:55:28.725627 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d8ab39-bc6f-4bc1-9dca-ecdcca04b89d None admin] RBAC: Adding query filter params (role.id=1f97e4602e2e4d6498913a81f9ae8d6a, scope.project.id=0e30ccf0c1204727b786876c141dbcd2, user.id=f29c5dfed31f42a9b5cbc0cd72e9fdc5) {{(pid=81423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 10:55:28.725933 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d8ab39-bc6f-4bc1-9dca-ecdcca04b89d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:55:28.727368 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d8ab39-bc6f-4bc1-9dca-ecdcca04b89d None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:55:28.733622 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 641/1287] 10.222.0.40 () {56 vars in 1374 bytes} [Fri Aug 26 10:55:28 2022] GET /identity/v3/role_assignments?user.id=f29c5dfed31f42a9b5cbc0cd72e9fdc5&scope.project.id=0e30ccf0c1204727b786876c141dbcd2&role.id=1f97e4602e2e4d6498913a81f9ae8d6a => generated 608 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 10:55:30.664996 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8cd64f-b446-46de-bcf7-d6775983121b None None] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:55:30.665520 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8cd64f-b446-46de-bcf7-d6775983121b None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:55:30.665520 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8cd64f-b446-46de-bcf7-d6775983121b None None] PATH_INFO: `/` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:55:30.666892 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 647/1288] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 26 10:55:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 10:55:30.672787 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97cd212b-8ea7-45ff-9ea3-6d82c559a99f None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:55:30.673261 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97cd212b-8ea7-45ff-9ea3-6d82c559a99f None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:55:30.673543 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97cd212b-8ea7-45ff-9ea3-6d82c559a99f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:55:30.692072 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-97cd212b-8ea7-45ff-9ea3-6d82c559a99f None None] MFA Rules not processed for user `ca155a48e2024c32af3fd2e19e5fac11`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 10:55:30.693262 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-97cd212b-8ea7-45ff-9ea3-6d82c559a99f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 10:55:30.722113 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:55:30.759787 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 642/1289] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 26 10:55:30 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 26 10:55:31.032297 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8cd64f-b446-46de-bcf7-d6775983121b None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:55:31.141968 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8cd64f-b446-46de-bcf7-d6775983121b None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:55:31.152759 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:55:31.177805 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6e519f-93e0-496c-9cdf-e2c384e81579 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:55:31.178464 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6e519f-93e0-496c-9cdf-e2c384e81579 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:55:31.178739 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6e519f-93e0-496c-9cdf-e2c384e81579 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:55:31.179008 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6e519f-93e0-496c-9cdf-e2c384e81579 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:55:31.179587 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb6e519f-93e0-496c-9cdf-e2c384e81579 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:55:31.180956 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb6e519f-93e0-496c-9cdf-e2c384e81579 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:55:31.189690 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application [None req-fb6e519f-93e0-496c-9cdf-e2c384e81579 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 26 10:55:31.189690 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 10:55:31.189690 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 26 10:55:31.189690 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 26 10:55:31.189690 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 26 10:55:31.189690 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 26 10:55:31.189690 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 26 10:55:31.189690 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 26 10:55:31.189690 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 26 10:55:31.189690 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Aug 26 10:55:31.189690 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 26 10:55:31.189690 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Aug 26 10:55:31.189690 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 10:55:31.189690 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 10:55:31.189690 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 10:55:31.189690 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 10:55:31.189690 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 10:55:31.194271 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 10:55:31.194271 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 10:55:31.194271 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 10:55:31.194271 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 10:55:31.194271 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 10:55:31.194271 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 10:55:31.194271 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 26 10:55:31.194271 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 26 10:55:31.194271 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 26 10:55:31.194271 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 26 10:55:31.194271 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 10:55:31.194271 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 10:55:31.194271 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 10:55:31.194271 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 10:55:31.194271 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 10:55:31.194271 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 10:55:31.195262 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 10:55:31.195262 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application with Lock( Aug 26 10:55:31.195262 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 10:55:31.195262 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._enter() Aug 26 10:55:31.195262 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 10:55:31.195262 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 10:55:31.195262 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 10:55:31.195262 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.creator() Aug 26 10:55:31.195262 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 10:55:31.195262 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application created_value = creator( Aug 26 10:55:31.195262 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 26 10:55:31.195262 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 26 10:55:31.195262 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 26 10:55:31.195262 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Aug 26 10:55:31.195262 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 648/1290] 10.222.0.40 () {56 vars in 1117 bytes} [Fri Aug 26 10:55:31 2022] GET /identity/v3/services/placement => generated 90 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 10:55:31.196838 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97cd212b-8ea7-45ff-9ea3-6d82c559a99f None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:55:31.304634 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97cd212b-8ea7-45ff-9ea3-6d82c559a99f None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:55:31.343909 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54de9ea1-f292-4069-9a1d-65971d6f8990 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:55:31.344520 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54de9ea1-f292-4069-9a1d-65971d6f8990 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:55:31.344798 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54de9ea1-f292-4069-9a1d-65971d6f8990 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:55:31.345066 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54de9ea1-f292-4069-9a1d-65971d6f8990 None admin] PATH_INFO: `/v3/services` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:55:31.345689 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54de9ea1-f292-4069-9a1d-65971d6f8990 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 10:55:31.346043 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54de9ea1-f292-4069-9a1d-65971d6f8990 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:55:31.347381 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54de9ea1-f292-4069-9a1d-65971d6f8990 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:55:31.352153 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 643/1291] 10.222.0.40 () {56 vars in 1116 bytes} [Fri Aug 26 10:55:31 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 10:55:31.358325 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6e519f-93e0-496c-9cdf-e2c384e81579 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:55:31.460572 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6e519f-93e0-496c-9cdf-e2c384e81579 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:55:31.493710 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561218e5-2397-4df0-ac2a-a30dc3144616 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:55:31.494328 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561218e5-2397-4df0-ac2a-a30dc3144616 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:55:31.494603 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561218e5-2397-4df0-ac2a-a30dc3144616 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:55:31.494870 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561218e5-2397-4df0-ac2a-a30dc3144616 None admin] PATH_INFO: `/v3/services` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:55:31.495487 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-561218e5-2397-4df0-ac2a-a30dc3144616 None admin] RBAC: Adding query filter params (type=placement) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 10:55:31.495780 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-561218e5-2397-4df0-ac2a-a30dc3144616 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:55:31.497126 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-561218e5-2397-4df0-ac2a-a30dc3144616 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:55:31.502263 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 649/1292] 10.222.0.40 () {56 vars in 1116 bytes} [Fri Aug 26 10:55:31 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 10:55:33.409145 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cff40b7-8bad-4dfe-ae09-bd8e56e17c00 None None] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:55:33.409657 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cff40b7-8bad-4dfe-ae09-bd8e56e17c00 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:55:33.409774 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cff40b7-8bad-4dfe-ae09-bd8e56e17c00 None None] PATH_INFO: `/` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:55:33.411072 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 644/1293] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 26 10:55:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 10:55:33.416162 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2747c64-6609-4c32-b6d4-626be6571010 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:55:33.416551 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2747c64-6609-4c32-b6d4-626be6571010 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:55:33.416716 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2747c64-6609-4c32-b6d4-626be6571010 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:55:33.434850 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-b2747c64-6609-4c32-b6d4-626be6571010 None None] MFA Rules not processed for user `ca155a48e2024c32af3fd2e19e5fac11`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 10:55:33.436147 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-b2747c64-6609-4c32-b6d4-626be6571010 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 10:55:33.465109 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:55:33.502511 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 650/1294] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 26 10:55:33 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 26 10:55:33.771855 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cff40b7-8bad-4dfe-ae09-bd8e56e17c00 None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:55:33.803534 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:55:33.873271 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cff40b7-8bad-4dfe-ae09-bd8e56e17c00 None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:55:33.906732 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e91a33f-031b-45cd-a41a-73b835699d48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:55:33.907330 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e91a33f-031b-45cd-a41a-73b835699d48 None admin] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:55:33.907606 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e91a33f-031b-45cd-a41a-73b835699d48 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:55:33.907875 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e91a33f-031b-45cd-a41a-73b835699d48 None admin] PATH_INFO: `/v3/services` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:55:33.908516 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e91a33f-031b-45cd-a41a-73b835699d48 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:55:33.909859 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e91a33f-031b-45cd-a41a-73b835699d48 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:55:33.924485 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 645/1295] 10.222.0.40 () {60 vars in 1141 bytes} [Fri Aug 26 10:55:33 2022] POST /identity/v3/services => generated 246 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 10:55:35.818579 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6647883-a86e-4eee-bfb2-d91885f3276c None None] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:55:35.819135 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6647883-a86e-4eee-bfb2-d91885f3276c None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:55:35.819135 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6647883-a86e-4eee-bfb2-d91885f3276c None None] PATH_INFO: `/` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:55:35.820456 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 651/1296] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 26 10:55:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 10:55:35.826410 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8a0e81-9584-4cfc-aea5-0bdff0d735ef None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:55:35.827165 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8a0e81-9584-4cfc-aea5-0bdff0d735ef None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:55:35.827165 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8a0e81-9584-4cfc-aea5-0bdff0d735ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:55:35.845091 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-de8a0e81-9584-4cfc-aea5-0bdff0d735ef None None] MFA Rules not processed for user `ca155a48e2024c32af3fd2e19e5fac11`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 10:55:35.846298 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-de8a0e81-9584-4cfc-aea5-0bdff0d735ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 10:55:35.916663 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 646/1297] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 26 10:55:35 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 26 10:55:36.182884 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6647883-a86e-4eee-bfb2-d91885f3276c None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:55:36.287288 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6647883-a86e-4eee-bfb2-d91885f3276c None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:55:36.319471 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80349a10-0743-4b61-8a1e-3f3c299cea50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:55:36.320088 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80349a10-0743-4b61-8a1e-3f3c299cea50 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:55:36.320365 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80349a10-0743-4b61-8a1e-3f3c299cea50 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:55:36.320631 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80349a10-0743-4b61-8a1e-3f3c299cea50 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:55:36.321201 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80349a10-0743-4b61-8a1e-3f3c299cea50 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:55:36.322600 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80349a10-0743-4b61-8a1e-3f3c299cea50 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:55:36.331761 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application [None req-80349a10-0743-4b61-8a1e-3f3c299cea50 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 26 10:55:36.331761 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 10:55:36.331761 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 26 10:55:36.331761 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 26 10:55:36.331761 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 26 10:55:36.331761 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 26 10:55:36.331761 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 26 10:55:36.331761 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 26 10:55:36.331761 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 26 10:55:36.331761 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Aug 26 10:55:36.331761 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 26 10:55:36.331761 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Aug 26 10:55:36.331761 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 10:55:36.331761 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 10:55:36.331761 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 10:55:36.331761 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 10:55:36.331761 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 10:55:36.334128 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 10:55:36.334128 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 10:55:36.334128 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 10:55:36.334128 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 10:55:36.334128 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 10:55:36.334128 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 10:55:36.334128 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 26 10:55:36.334128 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 26 10:55:36.334128 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 26 10:55:36.334128 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 26 10:55:36.334128 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 10:55:36.334128 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 10:55:36.334128 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 10:55:36.334128 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 10:55:36.334128 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 10:55:36.334128 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 10:55:36.335214 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 10:55:36.335214 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application with Lock( Aug 26 10:55:36.335214 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 10:55:36.335214 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._enter() Aug 26 10:55:36.335214 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 10:55:36.335214 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 10:55:36.335214 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 10:55:36.335214 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.creator() Aug 26 10:55:36.335214 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 10:55:36.335214 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application created_value = creator( Aug 26 10:55:36.335214 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 26 10:55:36.335214 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 26 10:55:36.335214 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 26 10:55:36.335214 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Aug 26 10:55:36.335214 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 652/1298] 10.222.0.40 () {56 vars in 1117 bytes} [Fri Aug 26 10:55:36 2022] GET /identity/v3/services/placement => generated 90 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 10:55:36.338855 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8a0e81-9584-4cfc-aea5-0bdff0d735ef None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:55:36.356643 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:55:36.437081 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8a0e81-9584-4cfc-aea5-0bdff0d735ef None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:55:36.470127 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb94d835-13ba-40a0-95cf-3f227bcb58c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:55:36.470720 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb94d835-13ba-40a0-95cf-3f227bcb58c1 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:55:36.470997 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb94d835-13ba-40a0-95cf-3f227bcb58c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:55:36.471265 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb94d835-13ba-40a0-95cf-3f227bcb58c1 None admin] PATH_INFO: `/v3/services` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:55:36.471893 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb94d835-13ba-40a0-95cf-3f227bcb58c1 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 10:55:36.472193 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb94d835-13ba-40a0-95cf-3f227bcb58c1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:55:36.473533 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb94d835-13ba-40a0-95cf-3f227bcb58c1 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:55:36.478541 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 647/1299] 10.222.0.40 () {56 vars in 1116 bytes} [Fri Aug 26 10:55:36 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 10:55:36.484903 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80349a10-0743-4b61-8a1e-3f3c299cea50 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:55:36.516284 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:55:36.585995 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80349a10-0743-4b61-8a1e-3f3c299cea50 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:55:36.619056 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa5c46b-a812-468d-ab5a-1bd18aeb533f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:55:36.619651 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa5c46b-a812-468d-ab5a-1bd18aeb533f None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:55:36.619928 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa5c46b-a812-468d-ab5a-1bd18aeb533f None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:55:36.620197 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa5c46b-a812-468d-ab5a-1bd18aeb533f None admin] PATH_INFO: `/v3/endpoints` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:55:36.620887 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa5c46b-a812-468d-ab5a-1bd18aeb533f None admin] RBAC: Adding query filter params (interface=public, service_id=49b34e10a37b44df84557e8ad8368515, region_id=RegionOne) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 10:55:36.621192 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa5c46b-a812-468d-ab5a-1bd18aeb533f None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:55:36.623158 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa5c46b-a812-468d-ab5a-1bd18aeb533f None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:55:36.628034 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 653/1300] 10.222.0.40 () {56 vars in 1251 bytes} [Fri Aug 26 10:55:36 2022] GET /identity/v3/endpoints?service_id=49b34e10a37b44df84557e8ad8368515&interface=public®ion_id=RegionOne => generated 194 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 10:55:36.633729 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb94d835-13ba-40a0-95cf-3f227bcb58c1 None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:55:36.719967 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:55:36.738869 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb94d835-13ba-40a0-95cf-3f227bcb58c1 None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:55:36.771778 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c908be4e-3247-4168-a288-6458dbe6a604 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:55:36.772372 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c908be4e-3247-4168-a288-6458dbe6a604 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:55:36.772643 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c908be4e-3247-4168-a288-6458dbe6a604 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:55:36.772907 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c908be4e-3247-4168-a288-6458dbe6a604 None admin] PATH_INFO: `/v3/services` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:55:36.773522 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c908be4e-3247-4168-a288-6458dbe6a604 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:55:36.774886 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c908be4e-3247-4168-a288-6458dbe6a604 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:55:36.782168 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 648/1301] 10.222.0.40 () {56 vars in 1087 bytes} [Fri Aug 26 10:55:36 2022] GET /identity/v3/services => generated 2191 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 10:55:38.645760 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d59963-3e3c-4605-bd66-3b7971c20e79 None None] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:55:38.646308 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d59963-3e3c-4605-bd66-3b7971c20e79 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:55:38.646403 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d59963-3e3c-4605-bd66-3b7971c20e79 None None] PATH_INFO: `/` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:55:38.647723 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 654/1302] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 26 10:55:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 10:55:38.652753 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b97980-152e-467e-9744-e5fb8665bced None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:55:38.653145 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b97980-152e-467e-9744-e5fb8665bced None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:55:38.653286 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b97980-152e-467e-9744-e5fb8665bced None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:55:38.672106 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-f0b97980-152e-467e-9744-e5fb8665bced None None] MFA Rules not processed for user `ca155a48e2024c32af3fd2e19e5fac11`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 10:55:38.673278 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-f0b97980-152e-467e-9744-e5fb8665bced None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 10:55:38.740286 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 649/1303] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 26 10:55:38 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 26 10:55:39.009136 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d59963-3e3c-4605-bd66-3b7971c20e79 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:55:39.104018 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:55:39.111405 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d59963-3e3c-4605-bd66-3b7971c20e79 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:55:39.144834 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5fc579-b5e2-4854-9d71-0d7b9e25bb9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:55:39.145452 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5fc579-b5e2-4854-9d71-0d7b9e25bb9e None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:55:39.145729 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5fc579-b5e2-4854-9d71-0d7b9e25bb9e None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:55:39.146016 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5fc579-b5e2-4854-9d71-0d7b9e25bb9e None admin] PATH_INFO: `/v3/services/placement` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:55:39.146600 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae5fc579-b5e2-4854-9d71-0d7b9e25bb9e None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:55:39.147967 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae5fc579-b5e2-4854-9d71-0d7b9e25bb9e None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:55:39.156704 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application [None req-ae5fc579-b5e2-4854-9d71-0d7b9e25bb9e None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 26 10:55:39.156704 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 10:55:39.156704 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 26 10:55:39.156704 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 26 10:55:39.156704 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 26 10:55:39.156704 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 26 10:55:39.156704 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 26 10:55:39.156704 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 26 10:55:39.156704 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 26 10:55:39.156704 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Aug 26 10:55:39.156704 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 26 10:55:39.156704 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Aug 26 10:55:39.156704 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 10:55:39.156704 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 10:55:39.156704 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 10:55:39.156704 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 10:55:39.156704 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 10:55:39.162138 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 10:55:39.162138 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 10:55:39.162138 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 10:55:39.162138 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 10:55:39.162138 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 10:55:39.162138 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 10:55:39.162138 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 26 10:55:39.162138 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 26 10:55:39.162138 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 26 10:55:39.162138 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 26 10:55:39.162138 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 10:55:39.162138 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 10:55:39.162138 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 10:55:39.162138 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 10:55:39.162138 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 10:55:39.162138 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 10:55:39.163209 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 10:55:39.163209 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application with Lock( Aug 26 10:55:39.163209 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 10:55:39.163209 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._enter() Aug 26 10:55:39.163209 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 10:55:39.163209 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 10:55:39.163209 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 10:55:39.163209 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.creator() Aug 26 10:55:39.163209 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 10:55:39.163209 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application created_value = creator( Aug 26 10:55:39.163209 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 26 10:55:39.163209 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 26 10:55:39.163209 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 26 10:55:39.163209 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Aug 26 10:55:39.163209 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 655/1304] 10.222.0.40 () {56 vars in 1117 bytes} [Fri Aug 26 10:55:39 2022] GET /identity/v3/services/placement => generated 90 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 10:55:39.164191 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b97980-152e-467e-9744-e5fb8665bced None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:55:39.239141 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:55:39.266961 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b97980-152e-467e-9744-e5fb8665bced None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:55:39.299786 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b230ae-de75-42fe-8c81-45b80f74fce0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:55:39.300381 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b230ae-de75-42fe-8c81-45b80f74fce0 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:55:39.300654 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b230ae-de75-42fe-8c81-45b80f74fce0 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:55:39.300920 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b230ae-de75-42fe-8c81-45b80f74fce0 None admin] PATH_INFO: `/v3/services` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:55:39.301534 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b230ae-de75-42fe-8c81-45b80f74fce0 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 10:55:39.301827 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b230ae-de75-42fe-8c81-45b80f74fce0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:55:39.303191 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b230ae-de75-42fe-8c81-45b80f74fce0 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:55:39.308111 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 650/1305] 10.222.0.40 () {56 vars in 1116 bytes} [Fri Aug 26 10:55:39 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 10:55:39.314800 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5fc579-b5e2-4854-9d71-0d7b9e25bb9e None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:55:39.411692 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5fc579-b5e2-4854-9d71-0d7b9e25bb9e None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:55:39.444585 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e04c224-e06f-4050-bdac-84cc84ec2379 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:55:39.445181 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e04c224-e06f-4050-bdac-84cc84ec2379 None admin] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:55:39.445456 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e04c224-e06f-4050-bdac-84cc84ec2379 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:55:39.445723 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e04c224-e06f-4050-bdac-84cc84ec2379 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:55:39.446403 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e04c224-e06f-4050-bdac-84cc84ec2379 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:55:39.447764 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e04c224-e06f-4050-bdac-84cc84ec2379 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:55:39.466739 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 656/1306] 10.222.0.40 () {60 vars in 1144 bytes} [Fri Aug 26 10:55:39 2022] POST /identity/v3/endpoints => generated 331 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 10:55:49.305327 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=30 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:55:49.462246 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:56:42.886403 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83526846-a40f-4aca-bb10-26d38e5787c8 None None] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:56:42.886927 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83526846-a40f-4aca-bb10-26d38e5787c8 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:56:42.886927 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83526846-a40f-4aca-bb10-26d38e5787c8 None None] PATH_INFO: `/` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:56:42.893030 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 651/1307] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 26 10:56:42 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 10:56:42.894572 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f098401-ce06-4aeb-867b-f2a1b22e6da4 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:56:42.895380 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f098401-ce06-4aeb-867b-f2a1b22e6da4 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:56:42.895862 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f098401-ce06-4aeb-867b-f2a1b22e6da4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:56:42.974881 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-4f098401-ce06-4aeb-867b-f2a1b22e6da4 None None] MFA Rules not processed for user `ca155a48e2024c32af3fd2e19e5fac11`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 10:56:42.976088 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-4f098401-ce06-4aeb-867b-f2a1b22e6da4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 10:56:43.115019 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 657/1308] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 26 10:56:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 10:56:43.413781 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91eeca1-18be-4db0-aa59-c622a7879f2b None None] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:56:43.414188 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91eeca1-18be-4db0-aa59-c622a7879f2b None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:56:43.414360 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91eeca1-18be-4db0-aa59-c622a7879f2b None None] PATH_INFO: `/` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:56:43.415856 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 652/1309] 10.222.0.40 () {54 vars in 965 bytes} [Fri Aug 26 10:56:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 10:56:43.422163 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4d0238-c49a-413d-91a0-9fb1619daa97 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:56:43.422541 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4d0238-c49a-413d-91a0-9fb1619daa97 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:56:43.422819 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4d0238-c49a-413d-91a0-9fb1619daa97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:56:43.505116 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-ad4d0238-c49a-413d-91a0-9fb1619daa97 None None] MFA Rules not processed for user `91da2acd2534423091537e1ba068d2b7`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 10:56:43.506396 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-ad4d0238-c49a-413d-91a0-9fb1619daa97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 10:56:43.529003 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:56:43.587695 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 658/1310] 10.222.0.40 () {58 vars in 1062 bytes} [Fri Aug 26 10:56:43 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 10:56:43.596792 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b91eeca1-18be-4db0-aa59-c622a7879f2b None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:56:43.598621 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-b91eeca1-18be-4db0-aa59-c622a7879f2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 10:56:43.716860 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b91eeca1-18be-4db0-aa59-c622a7879f2b None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:56:43.800487 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5d77a5-1b60-40a7-ad9b-ffc14cefd0a1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91da2acd2534423091537e1ba068d2b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e30ccf0c1204727b786876c141dbcd2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:56:43.801112 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5d77a5-1b60-40a7-ad9b-ffc14cefd0a1 service neutron] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:56:43.801391 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5d77a5-1b60-40a7-ad9b-ffc14cefd0a1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:56:43.801658 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5d77a5-1b60-40a7-ad9b-ffc14cefd0a1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:56:43.803394 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-8f5d77a5-1b60-40a7-ad9b-ffc14cefd0a1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 10:56:43.869836 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:56:43.933210 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f5d77a5-1b60-40a7-ad9b-ffc14cefd0a1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:56:43.934839 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f5d77a5-1b60-40a7-ad9b-ffc14cefd0a1 service neutron] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:56:43.957417 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 653/1311] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 10:56:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 361 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 10:56:46.246584 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e871fb79-a6a9-4df0-9885-d0241d2ae582 None None] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:56:46.247160 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e871fb79-a6a9-4df0-9885-d0241d2ae582 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:56:46.247160 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e871fb79-a6a9-4df0-9885-d0241d2ae582 None None] PATH_INFO: `/` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:56:46.248629 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 659/1312] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 26 10:56:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 10:56:46.254179 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e52a93-5109-4a07-baeb-15165f97dfd3 None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:56:46.254786 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e52a93-5109-4a07-baeb-15165f97dfd3 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:56:46.254786 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e52a93-5109-4a07-baeb-15165f97dfd3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:56:46.354838 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-78e52a93-5109-4a07-baeb-15165f97dfd3 None None] MFA Rules not processed for user `ca155a48e2024c32af3fd2e19e5fac11`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 10:56:46.356251 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-78e52a93-5109-4a07-baeb-15165f97dfd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 10:56:46.479862 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 654/1313] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 26 10:56:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 10:56:46.782488 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e871fb79-a6a9-4df0-9885-d0241d2ae582 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:56:46.784030 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-e871fb79-a6a9-4df0-9885-d0241d2ae582 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 10:56:46.888281 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e871fb79-a6a9-4df0-9885-d0241d2ae582 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:56:46.959411 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:56:46.962164 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c28dd0-4de7-4dde-bf8b-a116f01ef473 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91da2acd2534423091537e1ba068d2b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e30ccf0c1204727b786876c141dbcd2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:56:46.962942 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c28dd0-4de7-4dde-bf8b-a116f01ef473 service neutron] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:56:46.963301 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c28dd0-4de7-4dde-bf8b-a116f01ef473 service neutron] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:56:46.963745 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c28dd0-4de7-4dde-bf8b-a116f01ef473 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:56:46.966230 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-44c28dd0-4de7-4dde-bf8b-a116f01ef473 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 10:56:47.074383 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c28dd0-4de7-4dde-bf8b-a116f01ef473 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:56:47.075813 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c28dd0-4de7-4dde-bf8b-a116f01ef473 service neutron] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:56:47.093745 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 660/1314] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 10:56:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 10:56:49.342931 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254060ec-5a93-426a-b14e-ead62dc0a972 None None] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:56:49.343474 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254060ec-5a93-426a-b14e-ead62dc0a972 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:56:49.343474 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254060ec-5a93-426a-b14e-ead62dc0a972 None None] PATH_INFO: `/` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:56:49.344847 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 655/1315] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 26 10:56:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 10:56:49.350046 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5884f6-c387-4300-b74c-758aaff2e2f9 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:56:49.350432 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5884f6-c387-4300-b74c-758aaff2e2f9 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:56:49.350594 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5884f6-c387-4300-b74c-758aaff2e2f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:56:49.415056 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-5f5884f6-c387-4300-b74c-758aaff2e2f9 None None] MFA Rules not processed for user `514ea0d902e04c41a548d79008a6d68d`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 10:56:49.416284 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-5f5884f6-c387-4300-b74c-758aaff2e2f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 10:56:49.469582 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:56:49.515012 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 661/1316] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 26 10:56:49 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 10:56:49.808296 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254060ec-5a93-426a-b14e-ead62dc0a972 None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:56:49.810460 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-254060ec-5a93-426a-b14e-ead62dc0a972 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 10:56:49.820067 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:56:49.914838 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254060ec-5a93-426a-b14e-ead62dc0a972 None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:56:49.988253 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e8747c-fa0a-449e-b0c5-79088bf65eef service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91da2acd2534423091537e1ba068d2b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e30ccf0c1204727b786876c141dbcd2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:56:49.988852 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e8747c-fa0a-449e-b0c5-79088bf65eef service neutron] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:56:49.989122 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e8747c-fa0a-449e-b0c5-79088bf65eef service neutron] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:56:49.989385 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e8747c-fa0a-449e-b0c5-79088bf65eef service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:56:49.991003 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-97e8747c-fa0a-449e-b0c5-79088bf65eef service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 10:56:50.052750 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:56:50.103134 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e8747c-fa0a-449e-b0c5-79088bf65eef service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:56:50.104567 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e8747c-fa0a-449e-b0c5-79088bf65eef service neutron] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:56:50.125459 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 656/1317] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 10:56:49 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 10:56:52.611180 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23cd3e00-50ff-4f56-a093-e58c674366ad None None] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:56:52.611716 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23cd3e00-50ff-4f56-a093-e58c674366ad None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:56:52.611716 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23cd3e00-50ff-4f56-a093-e58c674366ad None None] PATH_INFO: `/` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:56:52.613304 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 662/1318] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 26 10:56:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 10:56:52.618456 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0ce133-25c7-46e9-a184-9e141e7f950e None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:56:52.618857 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0ce133-25c7-46e9-a184-9e141e7f950e None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:56:52.618995 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0ce133-25c7-46e9-a184-9e141e7f950e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:56:52.683055 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-9d0ce133-25c7-46e9-a184-9e141e7f950e None None] MFA Rules not processed for user `ca155a48e2024c32af3fd2e19e5fac11`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 10:56:52.684801 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-9d0ce133-25c7-46e9-a184-9e141e7f950e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 10:56:52.780946 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 657/1319] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 26 10:56:52 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 10:56:53.083140 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62df9b4d-f48a-41a6-addb-8b480e74946f None None] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:56:53.083441 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62df9b4d-f48a-41a6-addb-8b480e74946f None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:56:53.083714 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62df9b4d-f48a-41a6-addb-8b480e74946f None None] PATH_INFO: `/` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:56:53.085041 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 663/1320] 10.222.0.40 () {54 vars in 965 bytes} [Fri Aug 26 10:56:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 10:56:53.091118 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d33416-5b67-4682-9319-ef34aaf3de8b None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:56:53.091398 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d33416-5b67-4682-9319-ef34aaf3de8b None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:56:53.091666 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d33416-5b67-4682-9319-ef34aaf3de8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:56:53.113824 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:56:53.166258 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-b6d33416-5b67-4682-9319-ef34aaf3de8b None None] MFA Rules not processed for user `91da2acd2534423091537e1ba068d2b7`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 10:56:53.167382 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-b6d33416-5b67-4682-9319-ef34aaf3de8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 10:56:53.243097 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 658/1321] 10.222.0.40 () {58 vars in 1062 bytes} [Fri Aug 26 10:56:53 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 10:56:53.253003 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62df9b4d-f48a-41a6-addb-8b480e74946f None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:56:53.254859 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-62df9b4d-f48a-41a6-addb-8b480e74946f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 10:56:53.358371 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62df9b4d-f48a-41a6-addb-8b480e74946f None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:56:53.430338 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753d1473-321f-4c5c-83fb-54484b91f1c3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91da2acd2534423091537e1ba068d2b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e30ccf0c1204727b786876c141dbcd2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:56:53.430870 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753d1473-321f-4c5c-83fb-54484b91f1c3 service neutron] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:56:53.431143 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753d1473-321f-4c5c-83fb-54484b91f1c3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:56:53.431411 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753d1473-321f-4c5c-83fb-54484b91f1c3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:56:53.432997 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-753d1473-321f-4c5c-83fb-54484b91f1c3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 10:56:53.440882 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:56:53.540027 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-753d1473-321f-4c5c-83fb-54484b91f1c3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:56:53.541458 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-753d1473-321f-4c5c-83fb-54484b91f1c3 service neutron] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:56:53.559816 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 664/1322] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 10:56:53 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 10:56:56.576758 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56673d9-3a71-4b12-8785-3a2979b64fc5 None None] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:56:56.577345 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56673d9-3a71-4b12-8785-3a2979b64fc5 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:56:56.577345 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56673d9-3a71-4b12-8785-3a2979b64fc5 None None] PATH_INFO: `/` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:56:56.578723 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 659/1323] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 26 10:56:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 10:56:56.584020 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543717cf-4819-4476-a0eb-5200becead78 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:56:56.584456 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543717cf-4819-4476-a0eb-5200becead78 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:56:56.584678 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543717cf-4819-4476-a0eb-5200becead78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:56:56.649613 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-543717cf-4819-4476-a0eb-5200becead78 None None] MFA Rules not processed for user `ca155a48e2024c32af3fd2e19e5fac11`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 10:56:56.650830 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-543717cf-4819-4476-a0eb-5200becead78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 10:56:56.707847 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:56:56.748795 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 665/1324] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 26 10:56:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 10:56:57.041890 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56673d9-3a71-4b12-8785-3a2979b64fc5 None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:56:57.044681 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-a56673d9-3a71-4b12-8785-3a2979b64fc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 10:56:57.148767 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56673d9-3a71-4b12-8785-3a2979b64fc5 None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:56:57.161768 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:56:57.225529 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade23781-f84d-40a0-93f6-bd3980a60cc1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91da2acd2534423091537e1ba068d2b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e30ccf0c1204727b786876c141dbcd2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:56:57.226167 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade23781-f84d-40a0-93f6-bd3980a60cc1 service neutron] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:56:57.226445 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade23781-f84d-40a0-93f6-bd3980a60cc1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:56:57.226717 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade23781-f84d-40a0-93f6-bd3980a60cc1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:56:57.228367 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-ade23781-f84d-40a0-93f6-bd3980a60cc1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 10:56:57.338931 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade23781-f84d-40a0-93f6-bd3980a60cc1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:56:57.341179 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade23781-f84d-40a0-93f6-bd3980a60cc1 service neutron] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:56:57.360694 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 660/1325] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 10:56:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 10:56:59.341660 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7273969-78ae-4755-9af1-0e61070f48e9 None None] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:56:59.342200 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7273969-78ae-4755-9af1-0e61070f48e9 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:56:59.342200 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7273969-78ae-4755-9af1-0e61070f48e9 None None] PATH_INFO: `/` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:56:59.343488 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 666/1326] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 26 10:56:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 10:56:59.348916 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7aa66a2-64bf-4472-9397-ae38ed79cd27 None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:56:59.349347 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7aa66a2-64bf-4472-9397-ae38ed79cd27 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:56:59.349451 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7aa66a2-64bf-4472-9397-ae38ed79cd27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:56:59.400387 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:56:59.419254 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-a7aa66a2-64bf-4472-9397-ae38ed79cd27 None None] MFA Rules not processed for user `ca155a48e2024c32af3fd2e19e5fac11`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 10:56:59.420487 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-a7aa66a2-64bf-4472-9397-ae38ed79cd27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 10:56:59.526448 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 661/1327] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 26 10:56:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 10:56:59.811189 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7273969-78ae-4755-9af1-0e61070f48e9 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:56:59.812773 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-f7273969-78ae-4755-9af1-0e61070f48e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 10:56:59.930388 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7273969-78ae-4755-9af1-0e61070f48e9 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:57:00.006683 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c2c300-ba75-49fd-8d60-ded10475d531 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91da2acd2534423091537e1ba068d2b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e30ccf0c1204727b786876c141dbcd2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:57:00.007286 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c2c300-ba75-49fd-8d60-ded10475d531 service neutron] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:57:00.007559 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c2c300-ba75-49fd-8d60-ded10475d531 service neutron] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:57:00.007824 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c2c300-ba75-49fd-8d60-ded10475d531 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:57:00.009405 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-75c2c300-ba75-49fd-8d60-ded10475d531 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 10:57:00.025810 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:57:00.122247 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c2c300-ba75-49fd-8d60-ded10475d531 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:57:00.123686 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c2c300-ba75-49fd-8d60-ded10475d531 service neutron] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:57:00.142217 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 667/1328] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 10:56:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 10:57:02.132895 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a52d74-1917-456c-8ad3-113f947132cf None None] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:57:02.133459 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a52d74-1917-456c-8ad3-113f947132cf None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:57:02.133459 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a52d74-1917-456c-8ad3-113f947132cf None None] PATH_INFO: `/` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:57:02.135038 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 662/1329] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 26 10:57:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 10:57:02.141051 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700fc013-b82d-4449-b09f-a3f1cab33ef7 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:57:02.141458 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700fc013-b82d-4449-b09f-a3f1cab33ef7 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:57:02.141717 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700fc013-b82d-4449-b09f-a3f1cab33ef7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:57:02.206878 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-700fc013-b82d-4449-b09f-a3f1cab33ef7 None None] MFA Rules not processed for user `514ea0d902e04c41a548d79008a6d68d`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 10:57:02.208083 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-700fc013-b82d-4449-b09f-a3f1cab33ef7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 10:57:02.272093 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:57:02.305092 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 668/1330] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 26 10:57:02 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 10:57:02.595321 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a52d74-1917-456c-8ad3-113f947132cf None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:57:02.596907 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-58a52d74-1917-456c-8ad3-113f947132cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 10:57:02.703828 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a52d74-1917-456c-8ad3-113f947132cf None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:57:02.723582 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:57:02.781334 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd90e39-feb2-403a-968c-98124ac7e9f4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91da2acd2534423091537e1ba068d2b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e30ccf0c1204727b786876c141dbcd2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:57:02.781944 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd90e39-feb2-403a-968c-98124ac7e9f4 service neutron] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:57:02.782252 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd90e39-feb2-403a-968c-98124ac7e9f4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:57:02.782521 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd90e39-feb2-403a-968c-98124ac7e9f4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:57:02.784142 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-dbd90e39-feb2-403a-968c-98124ac7e9f4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 10:57:02.903359 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd90e39-feb2-403a-968c-98124ac7e9f4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:57:02.905035 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd90e39-feb2-403a-968c-98124ac7e9f4 service neutron] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:57:02.925383 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 663/1331] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 10:57:02 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 10:57:05.085602 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797e8748-394e-41a5-96da-24127ff8dd38 None None] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:57:05.086157 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797e8748-394e-41a5-96da-24127ff8dd38 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:57:05.086157 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797e8748-394e-41a5-96da-24127ff8dd38 None None] PATH_INFO: `/` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:57:05.087420 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 669/1332] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 26 10:57:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 10:57:05.092574 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41eb2777-9c4e-4889-abe0-d180393a7b79 None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:57:05.092998 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41eb2777-9c4e-4889-abe0-d180393a7b79 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:57:05.093178 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41eb2777-9c4e-4889-abe0-d180393a7b79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:57:05.156559 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:57:05.159740 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-41eb2777-9c4e-4889-abe0-d180393a7b79 None None] MFA Rules not processed for user `ca155a48e2024c32af3fd2e19e5fac11`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 10:57:05.161447 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-41eb2777-9c4e-4889-abe0-d180393a7b79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 10:57:05.266776 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 664/1333] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 26 10:57:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 10:57:05.548447 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797e8748-394e-41a5-96da-24127ff8dd38 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:57:05.550007 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-797e8748-394e-41a5-96da-24127ff8dd38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 10:57:05.653576 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797e8748-394e-41a5-96da-24127ff8dd38 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:57:05.725170 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f2edbb-928b-48ad-8cfb-eff086077ecf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91da2acd2534423091537e1ba068d2b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e30ccf0c1204727b786876c141dbcd2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:57:05.725768 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f2edbb-928b-48ad-8cfb-eff086077ecf service neutron] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:57:05.726074 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f2edbb-928b-48ad-8cfb-eff086077ecf service neutron] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:57:05.726349 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f2edbb-928b-48ad-8cfb-eff086077ecf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:57:05.727948 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-b4f2edbb-928b-48ad-8cfb-eff086077ecf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 10:57:05.754250 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:57:05.833962 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f2edbb-928b-48ad-8cfb-eff086077ecf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:57:05.835379 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f2edbb-928b-48ad-8cfb-eff086077ecf service neutron] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:57:05.854285 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 670/1334] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 10:57:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 10:57:07.798467 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa8d65d-9010-4510-8e14-9e4f9ef866c3 None None] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:57:07.799029 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa8d65d-9010-4510-8e14-9e4f9ef866c3 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:57:07.799029 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa8d65d-9010-4510-8e14-9e4f9ef866c3 None None] PATH_INFO: `/` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:57:07.800358 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 665/1335] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 26 10:57:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 10:57:07.805657 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0108c97-250d-4bcc-aa97-9a4535c607c5 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:57:07.806107 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0108c97-250d-4bcc-aa97-9a4535c607c5 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:57:07.806221 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0108c97-250d-4bcc-aa97-9a4535c607c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:57:07.872075 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-a0108c97-250d-4bcc-aa97-9a4535c607c5 None None] MFA Rules not processed for user `ca155a48e2024c32af3fd2e19e5fac11`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 10:57:07.873437 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-a0108c97-250d-4bcc-aa97-9a4535c607c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 10:57:07.955309 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:57:07.977140 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 671/1336] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 26 10:57:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 10:57:08.260527 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa8d65d-9010-4510-8e14-9e4f9ef866c3 None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:57:08.262637 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-5aa8d65d-9010-4510-8e14-9e4f9ef866c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 10:57:08.368781 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa8d65d-9010-4510-8e14-9e4f9ef866c3 None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:57:08.397504 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:57:08.441685 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2238dfe4-e0ee-4a48-bfe4-943bee48db18 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91da2acd2534423091537e1ba068d2b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e30ccf0c1204727b786876c141dbcd2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:57:08.442310 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2238dfe4-e0ee-4a48-bfe4-943bee48db18 service neutron] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:57:08.442580 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2238dfe4-e0ee-4a48-bfe4-943bee48db18 service neutron] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:57:08.442843 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2238dfe4-e0ee-4a48-bfe4-943bee48db18 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:57:08.444424 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-2238dfe4-e0ee-4a48-bfe4-943bee48db18 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 10:57:08.550127 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2238dfe4-e0ee-4a48-bfe4-943bee48db18 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:57:08.551481 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2238dfe4-e0ee-4a48-bfe4-943bee48db18 service neutron] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:57:08.569794 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 666/1337] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 10:57:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 10:57:10.818070 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d706eb48-9fc0-4e75-b77d-bc16aa0d781b None None] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:57:10.818623 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d706eb48-9fc0-4e75-b77d-bc16aa0d781b None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:57:10.818623 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d706eb48-9fc0-4e75-b77d-bc16aa0d781b None None] PATH_INFO: `/` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:57:10.819957 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 672/1338] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 26 10:57:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 10:57:10.825019 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b6d047-1432-44fe-bc16-746cee95a30d None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:57:10.826012 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b6d047-1432-44fe-bc16-746cee95a30d None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:57:10.826161 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b6d047-1432-44fe-bc16-746cee95a30d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:57:10.892129 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-d9b6d047-1432-44fe-bc16-746cee95a30d None None] MFA Rules not processed for user `ca155a48e2024c32af3fd2e19e5fac11`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 10:57:10.893343 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-d9b6d047-1432-44fe-bc16-746cee95a30d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 10:57:10.901628 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:57:10.994366 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 667/1339] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 26 10:57:10 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 10:57:11.281357 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d706eb48-9fc0-4e75-b77d-bc16aa0d781b None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:57:11.282924 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-d706eb48-9fc0-4e75-b77d-bc16aa0d781b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 10:57:11.385769 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d706eb48-9fc0-4e75-b77d-bc16aa0d781b None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:57:11.458883 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efafc9ef-33f3-4b6d-a4e4-2605504dd056 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91da2acd2534423091537e1ba068d2b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e30ccf0c1204727b786876c141dbcd2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:57:11.459483 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efafc9ef-33f3-4b6d-a4e4-2605504dd056 service neutron] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:57:11.459761 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efafc9ef-33f3-4b6d-a4e4-2605504dd056 service neutron] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:57:11.460031 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efafc9ef-33f3-4b6d-a4e4-2605504dd056 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:57:11.461636 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-efafc9ef-33f3-4b6d-a4e4-2605504dd056 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 10:57:11.497713 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:57:11.570508 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efafc9ef-33f3-4b6d-a4e4-2605504dd056 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:57:11.571876 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efafc9ef-33f3-4b6d-a4e4-2605504dd056 service neutron] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:57:11.589599 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 673/1340] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 10:57:11 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 10:57:15.558695 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd696c89-9a3d-410e-84a4-58fe6acaa0f4 None None] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:57:15.559204 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd696c89-9a3d-410e-84a4-58fe6acaa0f4 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:57:15.559378 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd696c89-9a3d-410e-84a4-58fe6acaa0f4 None None] PATH_INFO: `/` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:57:15.561290 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 668/1341] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 26 10:57:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 10:57:15.566575 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9f0f89-7e8a-440d-bf00-d1d51204e2e9 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:57:15.567022 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9f0f89-7e8a-440d-bf00-d1d51204e2e9 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:57:15.567119 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9f0f89-7e8a-440d-bf00-d1d51204e2e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:57:15.635542 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-5b9f0f89-7e8a-440d-bf00-d1d51204e2e9 None None] MFA Rules not processed for user `ca155a48e2024c32af3fd2e19e5fac11`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 10:57:15.636762 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-5b9f0f89-7e8a-440d-bf00-d1d51204e2e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 10:57:15.724932 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:57:15.737011 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 674/1342] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 26 10:57:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 10:57:16.023945 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd696c89-9a3d-410e-84a4-58fe6acaa0f4 None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:57:16.025478 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-fd696c89-9a3d-410e-84a4-58fe6acaa0f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 10:57:16.132150 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd696c89-9a3d-410e-84a4-58fe6acaa0f4 None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:57:16.171862 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:57:16.209959 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5796e313-e3d5-4eaf-93be-4f288dee1fc3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91da2acd2534423091537e1ba068d2b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e30ccf0c1204727b786876c141dbcd2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:57:16.210602 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5796e313-e3d5-4eaf-93be-4f288dee1fc3 service neutron] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:57:16.210867 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5796e313-e3d5-4eaf-93be-4f288dee1fc3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:57:16.211135 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5796e313-e3d5-4eaf-93be-4f288dee1fc3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:57:16.212750 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-5796e313-e3d5-4eaf-93be-4f288dee1fc3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 10:57:16.323777 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5796e313-e3d5-4eaf-93be-4f288dee1fc3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:57:16.325667 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5796e313-e3d5-4eaf-93be-4f288dee1fc3 service neutron] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:57:16.344609 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 669/1343] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 10:57:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 10:57:18.859417 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23160c4e-3478-42be-8373-a8f225817406 None None] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:57:18.859943 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23160c4e-3478-42be-8373-a8f225817406 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:57:18.859943 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23160c4e-3478-42be-8373-a8f225817406 None None] PATH_INFO: `/` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:57:18.861304 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 675/1344] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 26 10:57:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 10:57:18.867098 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c0d843-3a86-46cd-a9aa-04e042bbd8e3 None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:57:18.867590 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c0d843-3a86-46cd-a9aa-04e042bbd8e3 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:57:18.867682 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c0d843-3a86-46cd-a9aa-04e042bbd8e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:57:18.939053 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-27c0d843-3a86-46cd-a9aa-04e042bbd8e3 None None] MFA Rules not processed for user `ca155a48e2024c32af3fd2e19e5fac11`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 10:57:18.941003 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-27c0d843-3a86-46cd-a9aa-04e042bbd8e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 10:57:18.959781 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:57:19.040099 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 670/1345] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 26 10:57:18 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 10:57:19.323574 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23160c4e-3478-42be-8373-a8f225817406 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:57:19.325120 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-23160c4e-3478-42be-8373-a8f225817406 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 10:57:19.428287 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23160c4e-3478-42be-8373-a8f225817406 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:57:19.501184 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53da9009-f603-42eb-9a4e-db248ec7f7a5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91da2acd2534423091537e1ba068d2b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e30ccf0c1204727b786876c141dbcd2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:57:19.501788 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53da9009-f603-42eb-9a4e-db248ec7f7a5 service neutron] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:57:19.502088 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53da9009-f603-42eb-9a4e-db248ec7f7a5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:57:19.502363 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53da9009-f603-42eb-9a4e-db248ec7f7a5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:57:19.503945 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-53da9009-f603-42eb-9a4e-db248ec7f7a5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 10:57:19.549523 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:57:19.621517 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53da9009-f603-42eb-9a4e-db248ec7f7a5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:57:19.623047 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53da9009-f603-42eb-9a4e-db248ec7f7a5 service neutron] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:57:19.641254 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 676/1346] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 10:57:19 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 10:57:23.625741 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1d082f-6e31-4eb0-a364-b14150bea9e8 None None] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:57:23.626197 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1d082f-6e31-4eb0-a364-b14150bea9e8 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:57:23.626275 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1d082f-6e31-4eb0-a364-b14150bea9e8 None None] PATH_INFO: `/` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:57:23.628814 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 671/1347] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 26 10:57:23 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 26 10:57:23.635418 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7f64ab-1d5e-4e21-9bd6-c592b006041f None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:57:23.635844 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7f64ab-1d5e-4e21-9bd6-c592b006041f None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:57:23.635844 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7f64ab-1d5e-4e21-9bd6-c592b006041f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:57:23.718261 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-2a7f64ab-1d5e-4e21-9bd6-c592b006041f None None] MFA Rules not processed for user `ca155a48e2024c32af3fd2e19e5fac11`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 10:57:23.719771 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-2a7f64ab-1d5e-4e21-9bd6-c592b006041f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 10:57:23.821050 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:57:23.823815 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 677/1348] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 26 10:57:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 10:57:24.112844 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1d082f-6e31-4eb0-a364-b14150bea9e8 None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:57:24.114477 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-9a1d082f-6e31-4eb0-a364-b14150bea9e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 10:57:24.224029 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1d082f-6e31-4eb0-a364-b14150bea9e8 None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:57:24.276311 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:57:24.303044 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57b91e0-0fbf-4b9b-b0e0-900330c86b53 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91da2acd2534423091537e1ba068d2b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e30ccf0c1204727b786876c141dbcd2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:57:24.303650 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57b91e0-0fbf-4b9b-b0e0-900330c86b53 service neutron] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:57:24.303923 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57b91e0-0fbf-4b9b-b0e0-900330c86b53 service neutron] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:57:24.304188 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57b91e0-0fbf-4b9b-b0e0-900330c86b53 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:57:24.305784 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-b57b91e0-0fbf-4b9b-b0e0-900330c86b53 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 10:57:24.415107 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b57b91e0-0fbf-4b9b-b0e0-900330c86b53 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:57:24.416536 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b57b91e0-0fbf-4b9b-b0e0-900330c86b53 service neutron] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:57:24.435580 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 672/1349] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 10:57:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 10:57:29.329827 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95284aa-3a89-4410-b680-c27f73d2df56 None None] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:57:29.330425 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95284aa-3a89-4410-b680-c27f73d2df56 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:57:29.330527 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95284aa-3a89-4410-b680-c27f73d2df56 None None] PATH_INFO: `/` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:57:29.333732 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 678/1350] 10.222.0.40 () {54 vars in 921 bytes} [Fri Aug 26 10:57:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 10:57:29.340491 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a371ae19-0d50-4d4d-837d-d4dc42c19519 None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:57:29.341428 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a371ae19-0d50-4d4d-837d-d4dc42c19519 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:57:29.342067 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a371ae19-0d50-4d4d-837d-d4dc42c19519 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:57:29.435343 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-a371ae19-0d50-4d4d-837d-d4dc42c19519 None None] MFA Rules not processed for user `f29c5dfed31f42a9b5cbc0cd72e9fdc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 10:57:29.436786 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-a371ae19-0d50-4d4d-837d-d4dc42c19519 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 10:57:29.458315 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:57:29.604270 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 673/1351] 10.222.0.40 () {58 vars in 1018 bytes} [Fri Aug 26 10:57:29 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 267 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 10:57:33.863342 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a34b81-9959-4be7-b5b8-95bb966da628 None None] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:57:33.863852 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a34b81-9959-4be7-b5b8-95bb966da628 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:57:33.863968 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a34b81-9959-4be7-b5b8-95bb966da628 None None] PATH_INFO: `/` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:57:33.865218 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 679/1352] 10.222.0.40 () {54 vars in 921 bytes} [Fri Aug 26 10:57:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 10:57:33.873828 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20d7f29-8e55-4640-805f-34ede87521db None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:57:33.873828 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20d7f29-8e55-4640-805f-34ede87521db None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:57:33.873828 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20d7f29-8e55-4640-805f-34ede87521db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:57:33.953859 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-f20d7f29-8e55-4640-805f-34ede87521db None None] MFA Rules not processed for user `f29c5dfed31f42a9b5cbc0cd72e9fdc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 10:57:33.955332 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-f20d7f29-8e55-4640-805f-34ede87521db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 10:57:34.033648 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:57:34.078035 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 674/1353] 10.222.0.40 () {58 vars in 1018 bytes} [Fri Aug 26 10:57:33 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 10:57:35.419843 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ae5341-aa80-4732-88fc-1357a24e1f21 None None] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:57:35.420400 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ae5341-aa80-4732-88fc-1357a24e1f21 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:57:35.420652 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ae5341-aa80-4732-88fc-1357a24e1f21 None None] PATH_INFO: `/` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:57:35.422093 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 680/1354] 10.222.0.40 () {54 vars in 921 bytes} [Fri Aug 26 10:57:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 10:57:35.427303 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa0f682-186c-436a-85f2-f85e1138fcf7 None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:57:35.427716 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa0f682-186c-436a-85f2-f85e1138fcf7 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:57:35.427858 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa0f682-186c-436a-85f2-f85e1138fcf7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:57:35.505639 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-8fa0f682-186c-436a-85f2-f85e1138fcf7 None None] MFA Rules not processed for user `f29c5dfed31f42a9b5cbc0cd72e9fdc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 10:57:35.506877 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-8fa0f682-186c-436a-85f2-f85e1138fcf7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 10:57:35.615181 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 675/1355] 10.222.0.40 () {58 vars in 1018 bytes} [Fri Aug 26 10:57:35 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 10:57:38.946144 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f90774-c07d-43e9-a1a2-c6dd7b55208b None None] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:57:38.946632 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f90774-c07d-43e9-a1a2-c6dd7b55208b None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:57:38.946872 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f90774-c07d-43e9-a1a2-c6dd7b55208b None None] PATH_INFO: `/` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:57:38.948218 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 681/1356] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 26 10:57:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 10:57:38.959522 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f90d31a-8414-4441-920c-e648ddabcc30 None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:57:38.960327 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f90d31a-8414-4441-920c-e648ddabcc30 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:57:38.960772 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f90d31a-8414-4441-920c-e648ddabcc30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:57:39.001595 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:57:39.058048 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-7f90d31a-8414-4441-920c-e648ddabcc30 None None] MFA Rules not processed for user `ca155a48e2024c32af3fd2e19e5fac11`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 10:57:39.059810 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-7f90d31a-8414-4441-920c-e648ddabcc30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 10:57:39.175293 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 676/1357] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 26 10:57:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 10:57:39.483759 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd64abb-54b3-4b5d-a3f4-ae3a7eec0f1f None None] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:57:39.484280 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd64abb-54b3-4b5d-a3f4-ae3a7eec0f1f None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:57:39.484732 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd64abb-54b3-4b5d-a3f4-ae3a7eec0f1f None None] PATH_INFO: `/` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:57:39.486268 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 682/1358] 10.222.0.40 () {54 vars in 962 bytes} [Fri Aug 26 10:57:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 10:57:39.494088 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84647fd-cdc8-4d97-b7d4-5da825799439 None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:57:39.494640 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84647fd-cdc8-4d97-b7d4-5da825799439 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:57:39.495165 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84647fd-cdc8-4d97-b7d4-5da825799439 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:57:39.578039 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-c84647fd-cdc8-4d97-b7d4-5da825799439 None None] MFA Rules not processed for user `1f4b4a662db049ecacc865ecff10bdb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 10:57:39.579347 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-c84647fd-cdc8-4d97-b7d4-5da825799439 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 10:57:39.660264 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:57:39.740228 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 677/1359] 10.222.0.40 () {58 vars in 1059 bytes} [Fri Aug 26 10:57:39 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 247 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 10:57:39.746156 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd64abb-54b3-4b5d-a3f4-ae3a7eec0f1f None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:57:39.747796 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-ffd64abb-54b3-4b5d-a3f4-ae3a7eec0f1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 10:57:39.925453 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd64abb-54b3-4b5d-a3f4-ae3a7eec0f1f None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:57:40.005033 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:57:40.049282 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61fa91d2-7557-4107-93aa-a2951697ef21 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f4b4a662db049ecacc865ecff10bdb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e30ccf0c1204727b786876c141dbcd2', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:57:40.049893 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fa91d2-7557-4107-93aa-a2951697ef21 service nova] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:57:40.050202 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fa91d2-7557-4107-93aa-a2951697ef21 service nova] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:57:40.050483 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fa91d2-7557-4107-93aa-a2951697ef21 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:57:40.052412 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-61fa91d2-7557-4107-93aa-a2951697ef21 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 10:57:40.150226 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61fa91d2-7557-4107-93aa-a2951697ef21 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:57:40.152188 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61fa91d2-7557-4107-93aa-a2951697ef21 service nova] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:57:40.174166 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 683/1360] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 10:57:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 428 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 26 10:57:42.349048 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f038a84-b3fb-45f6-9d8c-276f59c3bd98 None None] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:57:42.349605 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f038a84-b3fb-45f6-9d8c-276f59c3bd98 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:57:42.349605 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f038a84-b3fb-45f6-9d8c-276f59c3bd98 None None] PATH_INFO: `/` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:57:42.351274 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 678/1361] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 26 10:57:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 10:57:42.356532 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8643b1ac-21a0-48bf-a20b-25c4c6ab81bf None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:57:42.357075 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8643b1ac-21a0-48bf-a20b-25c4c6ab81bf None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:57:42.357075 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8643b1ac-21a0-48bf-a20b-25c4c6ab81bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:57:42.428096 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-8643b1ac-21a0-48bf-a20b-25c4c6ab81bf None None] MFA Rules not processed for user `ca155a48e2024c32af3fd2e19e5fac11`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 10:57:42.429328 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-8643b1ac-21a0-48bf-a20b-25c4c6ab81bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 10:57:42.477591 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:57:42.555493 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 684/1362] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 26 10:57:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 10:57:42.852688 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f038a84-b3fb-45f6-9d8c-276f59c3bd98 None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:57:42.857337 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-9f038a84-b3fb-45f6-9d8c-276f59c3bd98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 10:57:43.012200 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f038a84-b3fb-45f6-9d8c-276f59c3bd98 None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:57:43.022269 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:57:43.146598 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396c56b5-a8a0-40e0-992d-9212d680b608 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f4b4a662db049ecacc865ecff10bdb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e30ccf0c1204727b786876c141dbcd2', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:57:43.147253 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396c56b5-a8a0-40e0-992d-9212d680b608 service nova] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:57:43.147485 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396c56b5-a8a0-40e0-992d-9212d680b608 service nova] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:57:43.147754 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396c56b5-a8a0-40e0-992d-9212d680b608 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:57:43.149362 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-396c56b5-a8a0-40e0-992d-9212d680b608 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 10:57:43.260154 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-396c56b5-a8a0-40e0-992d-9212d680b608 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:57:43.261888 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-396c56b5-a8a0-40e0-992d-9212d680b608 service nova] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:57:43.284004 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 679/1363] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 10:57:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 432 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 10:57:45.550583 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba6bceb-245e-40b0-9c07-23962b0d458a None None] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:57:45.551479 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba6bceb-245e-40b0-9c07-23962b0d458a None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:57:45.551911 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba6bceb-245e-40b0-9c07-23962b0d458a None None] PATH_INFO: `/` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:57:45.553429 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 685/1364] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 26 10:57:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 26 10:57:45.558865 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe877ec-4d07-4241-8e54-4f4ae8686bc2 None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:57:45.560144 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe877ec-4d07-4241-8e54-4f4ae8686bc2 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:57:45.560463 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe877ec-4d07-4241-8e54-4f4ae8686bc2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:57:45.577371 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:57:45.633864 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-ffe877ec-4d07-4241-8e54-4f4ae8686bc2 None None] MFA Rules not processed for user `ca155a48e2024c32af3fd2e19e5fac11`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 10:57:45.635099 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-ffe877ec-4d07-4241-8e54-4f4ae8686bc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 10:57:45.738150 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 680/1365] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 26 10:57:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 10:57:46.030064 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82857c4a-ab50-4f19-838c-d62f67d8a00f None None] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:57:46.030347 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82857c4a-ab50-4f19-838c-d62f67d8a00f None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:57:46.030612 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82857c4a-ab50-4f19-838c-d62f67d8a00f None None] PATH_INFO: `/` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:57:46.032082 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 686/1366] 10.222.0.40 () {54 vars in 962 bytes} [Fri Aug 26 10:57:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 10:57:46.037291 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838e0ee5-6ee9-400c-bd53-eac6031e9126 None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:57:46.037664 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838e0ee5-6ee9-400c-bd53-eac6031e9126 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:57:46.038071 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838e0ee5-6ee9-400c-bd53-eac6031e9126 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:57:46.124354 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-838e0ee5-6ee9-400c-bd53-eac6031e9126 None None] MFA Rules not processed for user `1f4b4a662db049ecacc865ecff10bdb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 10:57:46.125559 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-838e0ee5-6ee9-400c-bd53-eac6031e9126 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 10:57:46.169383 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:57:46.261390 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 681/1367] 10.222.0.40 () {58 vars in 1059 bytes} [Fri Aug 26 10:57:46 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 10:57:46.269626 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82857c4a-ab50-4f19-838c-d62f67d8a00f None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:57:46.271257 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-82857c4a-ab50-4f19-838c-d62f67d8a00f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 10:57:46.433398 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82857c4a-ab50-4f19-838c-d62f67d8a00f None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:57:46.454543 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:57:46.557947 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4224cdc-c335-4d51-b96d-9272be235040 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f4b4a662db049ecacc865ecff10bdb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e30ccf0c1204727b786876c141dbcd2', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:57:46.558679 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4224cdc-c335-4d51-b96d-9272be235040 service nova] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:57:46.559117 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4224cdc-c335-4d51-b96d-9272be235040 service nova] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:57:46.559555 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4224cdc-c335-4d51-b96d-9272be235040 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:57:46.561269 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-d4224cdc-c335-4d51-b96d-9272be235040 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 10:57:46.657101 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4224cdc-c335-4d51-b96d-9272be235040 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:57:46.659741 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4224cdc-c335-4d51-b96d-9272be235040 service nova] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:57:46.678235 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 687/1368] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 10:57:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 409 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 10:57:47.156722 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078bc297-e915-4472-9b4b-4af9b085b46c None None] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:57:47.157258 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078bc297-e915-4472-9b4b-4af9b085b46c None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:57:47.157258 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078bc297-e915-4472-9b4b-4af9b085b46c None None] PATH_INFO: `/` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:57:47.158805 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 682/1369] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 26 10:57:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 10:57:47.163930 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cdf232e-6b7f-437e-b4fa-c654ee0f7a10 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:57:47.164213 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cdf232e-6b7f-437e-b4fa-c654ee0f7a10 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:57:47.164475 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cdf232e-6b7f-437e-b4fa-c654ee0f7a10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:57:47.199222 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:57:47.240658 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-2cdf232e-6b7f-437e-b4fa-c654ee0f7a10 None None] MFA Rules not processed for user `ca155a48e2024c32af3fd2e19e5fac11`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 10:57:47.242071 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-2cdf232e-6b7f-437e-b4fa-c654ee0f7a10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 10:57:47.345625 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 688/1370] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 26 10:57:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 10:57:47.971018 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be0ae9c-36ee-492e-9feb-cc4510943cd9 None None] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:57:47.971593 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be0ae9c-36ee-492e-9feb-cc4510943cd9 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:57:47.972020 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be0ae9c-36ee-492e-9feb-cc4510943cd9 None None] PATH_INFO: `/` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:57:47.974246 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 683/1371] 10.222.0.40 () {54 vars in 964 bytes} [Fri Aug 26 10:57:47 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 10:57:47.980666 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc81699-a853-430b-94be-357dba7e35fd None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:57:47.982484 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc81699-a853-430b-94be-357dba7e35fd None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:57:47.982981 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc81699-a853-430b-94be-357dba7e35fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:57:48.082103 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-0cc81699-a853-430b-94be-357dba7e35fd None None] MFA Rules not processed for user `c277338a7f814cc6b01c5419775031bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 10:57:48.083321 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-0cc81699-a853-430b-94be-357dba7e35fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 10:57:48.140214 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:57:48.179023 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 689/1372] 10.222.0.40 () {58 vars in 1061 bytes} [Fri Aug 26 10:57:47 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 10:57:48.187417 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be0ae9c-36ee-492e-9feb-cc4510943cd9 None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:57:48.189356 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-7be0ae9c-36ee-492e-9feb-cc4510943cd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 10:57:48.328532 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be0ae9c-36ee-492e-9feb-cc4510943cd9 None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:57:48.394806 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:57:48.441874 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a965c0af-2144-448f-bead-471fe1be72bf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c277338a7f814cc6b01c5419775031bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e30ccf0c1204727b786876c141dbcd2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:57:48.442632 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a965c0af-2144-448f-bead-471fe1be72bf service cinder] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:57:48.442901 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a965c0af-2144-448f-bead-471fe1be72bf service cinder] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:57:48.443836 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a965c0af-2144-448f-bead-471fe1be72bf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:57:48.444847 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-a965c0af-2144-448f-bead-471fe1be72bf service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 10:57:48.579275 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a965c0af-2144-448f-bead-471fe1be72bf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:57:48.580856 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a965c0af-2144-448f-bead-471fe1be72bf service cinder] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:57:48.610586 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 684/1373] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 10:57:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 420 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 10:57:48.807176 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c2014d-93c9-40d1-a70e-81e619f67656 None None] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:57:48.807463 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c2014d-93c9-40d1-a70e-81e619f67656 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:57:48.807741 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c2014d-93c9-40d1-a70e-81e619f67656 None None] PATH_INFO: `/` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:57:48.809693 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 690/1374] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 26 10:57:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 10:57:48.814968 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e658ca-adf0-42a9-807e-5334b7262d84 None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:57:48.815251 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e658ca-adf0-42a9-807e-5334b7262d84 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:57:48.815526 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e658ca-adf0-42a9-807e-5334b7262d84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:57:48.886791 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-89e658ca-adf0-42a9-807e-5334b7262d84 None None] MFA Rules not processed for user `ca155a48e2024c32af3fd2e19e5fac11`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 10:57:48.888826 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-89e658ca-adf0-42a9-807e-5334b7262d84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 10:57:48.909494 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:57:48.989029 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 685/1375] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 26 10:57:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 10:57:49.280874 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c2014d-93c9-40d1-a70e-81e619f67656 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:57:49.282483 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-79c2014d-93c9-40d1-a70e-81e619f67656 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 10:57:49.431410 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c2014d-93c9-40d1-a70e-81e619f67656 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:57:49.482971 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:57:49.545912 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7844d732-540a-4589-bbc3-652ba8ac2d18 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f4b4a662db049ecacc865ecff10bdb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e30ccf0c1204727b786876c141dbcd2', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:57:49.546470 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7844d732-540a-4589-bbc3-652ba8ac2d18 service nova] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:57:49.546751 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7844d732-540a-4589-bbc3-652ba8ac2d18 service nova] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:57:49.547013 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7844d732-540a-4589-bbc3-652ba8ac2d18 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:57:49.548622 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-7844d732-540a-4589-bbc3-652ba8ac2d18 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 10:57:49.642516 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7844d732-540a-4589-bbc3-652ba8ac2d18 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:57:49.644115 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7844d732-540a-4589-bbc3-652ba8ac2d18 service nova] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:57:49.662656 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 691/1376] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 10:57:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 382 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 10:57:51.603549 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42683d19-5c5a-4427-b41b-c2e857599dbb None None] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:57:51.604013 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42683d19-5c5a-4427-b41b-c2e857599dbb None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:57:51.604110 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42683d19-5c5a-4427-b41b-c2e857599dbb None None] PATH_INFO: `/` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:57:51.605430 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 686/1377] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 26 10:57:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 10:57:51.611127 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb2d410-e386-4dee-81b5-c7560a818a9a None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:57:51.611537 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb2d410-e386-4dee-81b5-c7560a818a9a None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:57:51.611664 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb2d410-e386-4dee-81b5-c7560a818a9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:57:51.695086 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-6eb2d410-e386-4dee-81b5-c7560a818a9a None None] MFA Rules not processed for user `ca155a48e2024c32af3fd2e19e5fac11`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 10:57:51.696286 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-6eb2d410-e386-4dee-81b5-c7560a818a9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 10:57:51.703162 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:57:51.795983 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 692/1378] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 26 10:57:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 10:57:52.093500 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42683d19-5c5a-4427-b41b-c2e857599dbb None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:57:52.098465 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-42683d19-5c5a-4427-b41b-c2e857599dbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 10:57:52.292534 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42683d19-5c5a-4427-b41b-c2e857599dbb None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:57:52.297012 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:57:52.423644 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66b91e2-2595-4e64-9bbf-d06a32e822f8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f4b4a662db049ecacc865ecff10bdb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e30ccf0c1204727b786876c141dbcd2', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:57:52.424256 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66b91e2-2595-4e64-9bbf-d06a32e822f8 service nova] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:57:52.424529 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66b91e2-2595-4e64-9bbf-d06a32e822f8 service nova] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:57:52.424796 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66b91e2-2595-4e64-9bbf-d06a32e822f8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:57:52.426775 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-e66b91e2-2595-4e64-9bbf-d06a32e822f8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 10:57:52.525161 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e66b91e2-2595-4e64-9bbf-d06a32e822f8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:57:52.526790 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e66b91e2-2595-4e64-9bbf-d06a32e822f8 service nova] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:57:52.547673 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 687/1379] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 10:57:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 454 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 10:57:54.550228 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d8019f-5d75-40b3-a3ad-41d11cc30557 None None] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:57:54.550756 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d8019f-5d75-40b3-a3ad-41d11cc30557 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:57:54.550756 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d8019f-5d75-40b3-a3ad-41d11cc30557 None None] PATH_INFO: `/` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:57:54.552272 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 693/1380] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 26 10:57:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 10:57:54.557446 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f6b090-10c3-4d32-ad13-87bbae31a364 None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:57:54.557920 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f6b090-10c3-4d32-ad13-87bbae31a364 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:57:54.558229 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f6b090-10c3-4d32-ad13-87bbae31a364 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:57:54.572204 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:57:54.632957 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-b0f6b090-10c3-4d32-ad13-87bbae31a364 None None] MFA Rules not processed for user `ca155a48e2024c32af3fd2e19e5fac11`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 10:57:54.634370 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-b0f6b090-10c3-4d32-ad13-87bbae31a364 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 10:57:54.750142 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 688/1381] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 26 10:57:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 10:57:55.036189 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d8019f-5d75-40b3-a3ad-41d11cc30557 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:57:55.037862 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-52d8019f-5d75-40b3-a3ad-41d11cc30557 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 10:57:55.206707 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:57:55.219791 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d8019f-5d75-40b3-a3ad-41d11cc30557 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:57:55.345020 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c482459b-9d9d-4de5-b4d1-abf54fc53bd8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f4b4a662db049ecacc865ecff10bdb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e30ccf0c1204727b786876c141dbcd2', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:57:55.345537 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c482459b-9d9d-4de5-b4d1-abf54fc53bd8 service nova] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:57:55.345821 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c482459b-9d9d-4de5-b4d1-abf54fc53bd8 service nova] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:57:55.346102 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c482459b-9d9d-4de5-b4d1-abf54fc53bd8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:57:55.347661 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-c482459b-9d9d-4de5-b4d1-abf54fc53bd8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 10:57:55.440532 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c482459b-9d9d-4de5-b4d1-abf54fc53bd8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:57:55.442115 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c482459b-9d9d-4de5-b4d1-abf54fc53bd8 service nova] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:57:55.453171 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:57:55.462392 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 694/1382] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 10:57:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 427 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 26 10:57:57.456841 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9b2d26-12a1-4e84-a31f-fab326c9edda None None] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:57:57.457437 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9b2d26-12a1-4e84-a31f-fab326c9edda None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:57:57.457437 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9b2d26-12a1-4e84-a31f-fab326c9edda None None] PATH_INFO: `/` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:57:57.458928 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 689/1383] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 26 10:57:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 10:57:57.464083 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6113d144-aab2-4a29-a5a2-8f85e4b5f3d8 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:57:57.464499 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6113d144-aab2-4a29-a5a2-8f85e4b5f3d8 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:57:57.464716 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6113d144-aab2-4a29-a5a2-8f85e4b5f3d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:57:57.535140 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-6113d144-aab2-4a29-a5a2-8f85e4b5f3d8 None None] MFA Rules not processed for user `ca155a48e2024c32af3fd2e19e5fac11`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 10:57:57.536398 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-6113d144-aab2-4a29-a5a2-8f85e4b5f3d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 10:57:57.636325 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 695/1384] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 26 10:57:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 10:57:58.005745 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9b2d26-12a1-4e84-a31f-fab326c9edda None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:57:58.007306 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-dc9b2d26-12a1-4e84-a31f-fab326c9edda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 10:57:58.104613 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:57:58.171868 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9b2d26-12a1-4e84-a31f-fab326c9edda None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:57:58.293022 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b9f220-bb5e-4bc1-988a-b620f32b2857 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f4b4a662db049ecacc865ecff10bdb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e30ccf0c1204727b786876c141dbcd2', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:57:58.293669 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b9f220-bb5e-4bc1-988a-b620f32b2857 service nova] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:57:58.294045 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b9f220-bb5e-4bc1-988a-b620f32b2857 service nova] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:57:58.294376 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b9f220-bb5e-4bc1-988a-b620f32b2857 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:57:58.296063 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-10b9f220-bb5e-4bc1-988a-b620f32b2857 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 10:57:58.349190 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:57:58.392351 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b9f220-bb5e-4bc1-988a-b620f32b2857 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:57:58.394024 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b9f220-bb5e-4bc1-988a-b620f32b2857 service nova] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:57:58.413418 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 690/1385] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 10:57:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 408 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 10:58:00.462057 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9550ad3-ced2-4327-b1fc-abd65fa2240e None None] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:58:00.462682 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9550ad3-ced2-4327-b1fc-abd65fa2240e None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:58:00.462682 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9550ad3-ced2-4327-b1fc-abd65fa2240e None None] PATH_INFO: `/` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:58:00.463939 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 696/1386] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 26 10:58:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 26 10:58:00.469433 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ba7d02-a6bc-40ff-a1f3-cd0a9f2a3cf8 None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:58:00.469924 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ba7d02-a6bc-40ff-a1f3-cd0a9f2a3cf8 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:58:00.470036 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ba7d02-a6bc-40ff-a1f3-cd0a9f2a3cf8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:58:00.711524 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-02ba7d02-a6bc-40ff-a1f3-cd0a9f2a3cf8 None None] MFA Rules not processed for user `ca155a48e2024c32af3fd2e19e5fac11`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 10:58:00.713388 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-02ba7d02-a6bc-40ff-a1f3-cd0a9f2a3cf8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 10:58:00.828096 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 691/1387] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 26 10:58:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 360 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 10:58:01.119549 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9550ad3-ced2-4327-b1fc-abd65fa2240e None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:58:01.122508 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-e9550ad3-ced2-4327-b1fc-abd65fa2240e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 10:58:01.212188 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:58:01.294918 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9550ad3-ced2-4327-b1fc-abd65fa2240e None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:58:01.440784 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2856d099-cd65-4134-b3ac-863ce00d9f70 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f4b4a662db049ecacc865ecff10bdb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e30ccf0c1204727b786876c141dbcd2', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:58:01.441398 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2856d099-cd65-4134-b3ac-863ce00d9f70 service nova] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:58:01.441667 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2856d099-cd65-4134-b3ac-863ce00d9f70 service nova] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:58:01.441938 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2856d099-cd65-4134-b3ac-863ce00d9f70 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:58:01.443587 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-2856d099-cd65-4134-b3ac-863ce00d9f70 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 10:58:01.493911 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:58:01.551550 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2856d099-cd65-4134-b3ac-863ce00d9f70 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:58:01.553113 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2856d099-cd65-4134-b3ac-863ce00d9f70 service nova] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:58:01.571886 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 697/1388] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 10:58:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 453 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 10:58:03.520679 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb70d90-3f66-4953-b7d7-fedd64f375ca None None] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:58:03.521246 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb70d90-3f66-4953-b7d7-fedd64f375ca None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:58:03.521246 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb70d90-3f66-4953-b7d7-fedd64f375ca None None] PATH_INFO: `/` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:58:03.522647 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 692/1389] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 26 10:58:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 10:58:03.528509 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b4bb6b-321f-4dc0-a268-18f885e5987f None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:58:03.528935 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b4bb6b-321f-4dc0-a268-18f885e5987f None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:58:03.529147 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b4bb6b-321f-4dc0-a268-18f885e5987f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:58:03.596352 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-45b4bb6b-321f-4dc0-a268-18f885e5987f None None] MFA Rules not processed for user `ca155a48e2024c32af3fd2e19e5fac11`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 10:58:03.597565 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-45b4bb6b-321f-4dc0-a268-18f885e5987f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 10:58:03.693207 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 698/1390] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 26 10:58:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 10:58:03.734944 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01468c5d-1681-44b9-8339-c036d3df7f0c None None] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:58:03.735223 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01468c5d-1681-44b9-8339-c036d3df7f0c None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:58:03.735492 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01468c5d-1681-44b9-8339-c036d3df7f0c None None] PATH_INFO: `/` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:58:03.736830 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 693/1391] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 26 10:58:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 10:58:03.742491 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f97d71-4125-4aa2-bc83-d8e33160fb40 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:58:03.742772 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f97d71-4125-4aa2-bc83-d8e33160fb40 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:58:03.743059 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f97d71-4125-4aa2-bc83-d8e33160fb40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:58:03.758620 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:58:03.810953 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-55f97d71-4125-4aa2-bc83-d8e33160fb40 None None] MFA Rules not processed for user `ca155a48e2024c32af3fd2e19e5fac11`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 10:58:03.812079 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-55f97d71-4125-4aa2-bc83-d8e33160fb40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 10:58:03.910159 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 699/1392] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 26 10:58:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 10:58:03.983086 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01468c5d-1681-44b9-8339-c036d3df7f0c None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:58:03.984661 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-01468c5d-1681-44b9-8339-c036d3df7f0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 10:58:04.017136 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:58:04.133670 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01468c5d-1681-44b9-8339-c036d3df7f0c None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:58:04.213211 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ca4478-8d2a-44d1-ba55-604d5ccdd574 None None] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:58:04.213497 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ca4478-8d2a-44d1-ba55-604d5ccdd574 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:58:04.213772 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ca4478-8d2a-44d1-ba55-604d5ccdd574 None None] PATH_INFO: `/` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:58:04.215817 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 700/1393] 10.222.0.40 () {54 vars in 967 bytes} [Fri Aug 26 10:58:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 26 10:58:04.220504 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd2860d-7974-4743-adef-cb01f5ae3e35 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:58:04.220787 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd2860d-7974-4743-adef-cb01f5ae3e35 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:58:04.221066 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd2860d-7974-4743-adef-cb01f5ae3e35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:58:04.240837 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:58:04.254583 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60373bb-be60-4a9e-a465-3b8ab953a43b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f4b4a662db049ecacc865ecff10bdb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e30ccf0c1204727b786876c141dbcd2', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:58:04.255198 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60373bb-be60-4a9e-a465-3b8ab953a43b service nova] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:58:04.255467 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60373bb-be60-4a9e-a465-3b8ab953a43b service nova] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:58:04.255742 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60373bb-be60-4a9e-a465-3b8ab953a43b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:58:04.257345 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-a60373bb-be60-4a9e-a465-3b8ab953a43b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 10:58:04.354744 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a60373bb-be60-4a9e-a465-3b8ab953a43b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:58:04.357091 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a60373bb-be60-4a9e-a465-3b8ab953a43b service nova] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:58:04.377890 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 694/1394] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 10:58:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 395 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 10:58:04.420304 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-8bd2860d-7974-4743-adef-cb01f5ae3e35 None None] MFA Rules not processed for user `8631d7ecfd2849daa8694aecccd22a05`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 10:58:04.421534 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-8bd2860d-7974-4743-adef-cb01f5ae3e35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 10:58:04.458605 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:58:04.508059 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 701/1395] 10.222.0.40 () {58 vars in 1064 bytes} [Fri Aug 26 10:58:04 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 288 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 10:58:04.517895 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60373bb-be60-4a9e-a465-3b8ab953a43b service nova] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:58:04.519252 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-a60373bb-be60-4a9e-a465-3b8ab953a43b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 10:58:04.599426 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60373bb-be60-4a9e-a465-3b8ab953a43b service nova] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:58:04.641311 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:58:04.675933 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e539f5f-f69c-45e9-a319-bba80262901d service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8631d7ecfd2849daa8694aecccd22a05', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e30ccf0c1204727b786876c141dbcd2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:58:04.676460 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e539f5f-f69c-45e9-a319-bba80262901d service glance] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:58:04.676750 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e539f5f-f69c-45e9-a319-bba80262901d service glance] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:58:04.676998 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e539f5f-f69c-45e9-a319-bba80262901d service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:58:04.678633 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-8e539f5f-f69c-45e9-a319-bba80262901d service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 10:58:04.789269 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e539f5f-f69c-45e9-a319-bba80262901d service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:58:04.790733 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e539f5f-f69c-45e9-a319-bba80262901d service glance] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:58:04.809819 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 695/1396] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 10:58:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 10:58:05.572490 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fde7164-4705-44ab-9a0c-d50f163210c7 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:58:05.572937 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fde7164-4705-44ab-9a0c-d50f163210c7 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:58:05.573040 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fde7164-4705-44ab-9a0c-d50f163210c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:58:05.639696 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-7fde7164-4705-44ab-9a0c-d50f163210c7 None None] MFA Rules not processed for user `50be12ccbd274a8792615e9dd10b61ca`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 10:58:05.640911 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-7fde7164-4705-44ab-9a0c-d50f163210c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 10:58:05.734161 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 702/1397] 10.222.0.40 () {58 vars in 1009 bytes} [Fri Aug 26 10:58:05 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 10:58:05.786750 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf6966b-275a-4ac5-ba11-dd7a139d8670 None None] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:58:05.787675 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf6966b-275a-4ac5-ba11-dd7a139d8670 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:58:05.788070 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf6966b-275a-4ac5-ba11-dd7a139d8670 None None] PATH_INFO: `/` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:58:05.789583 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 696/1398] 10.222.0.40 () {54 vars in 943 bytes} [Fri Aug 26 10:58:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 10:58:05.796853 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571a1694-2a5b-4a0e-be32-728d41639f37 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:58:05.797145 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571a1694-2a5b-4a0e-be32-728d41639f37 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:58:05.797403 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571a1694-2a5b-4a0e-be32-728d41639f37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:58:05.868575 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:58:05.891642 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-571a1694-2a5b-4a0e-be32-728d41639f37 None None] MFA Rules not processed for user `6529bea296c9482a973ced5fe9f2212f`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 10:58:05.892941 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-571a1694-2a5b-4a0e-be32-728d41639f37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 10:58:06.013697 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 703/1399] 10.222.0.40 () {58 vars in 1040 bytes} [Fri Aug 26 10:58:05 2022] POST /identity/v3/auth/tokens => generated 3304 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 10:58:06.023105 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf6966b-275a-4ac5-ba11-dd7a139d8670 None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:58:06.024617 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-edf6966b-275a-4ac5-ba11-dd7a139d8670 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 10:58:06.101305 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:58:06.163164 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf6966b-275a-4ac5-ba11-dd7a139d8670 None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:58:06.271010 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67312813-274a-4b9d-ae1f-56cd348d1e1d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6529bea296c9482a973ced5fe9f2212f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e30ccf0c1204727b786876c141dbcd2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:58:06.271613 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67312813-274a-4b9d-ae1f-56cd348d1e1d service swift] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:58:06.271882 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67312813-274a-4b9d-ae1f-56cd348d1e1d service swift] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:58:06.272142 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67312813-274a-4b9d-ae1f-56cd348d1e1d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:58:06.273838 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-67312813-274a-4b9d-ae1f-56cd348d1e1d service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 10:58:06.332705 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:58:06.354396 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0820f472-a119-486b-8ddb-56bf4affa348 None None] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:58:06.354672 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0820f472-a119-486b-8ddb-56bf4affa348 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:58:06.354939 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0820f472-a119-486b-8ddb-56bf4affa348 None None] PATH_INFO: `/` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:58:06.356390 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 704/1400] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 26 10:58:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 26 10:58:06.357870 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67312813-274a-4b9d-ae1f-56cd348d1e1d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:58:06.359486 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67312813-274a-4b9d-ae1f-56cd348d1e1d service swift] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:58:06.361371 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50640a74-0e29-4035-b9d9-1738f88d49fc None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:58:06.361704 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50640a74-0e29-4035-b9d9-1738f88d49fc None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:58:06.362095 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50640a74-0e29-4035-b9d9-1738f88d49fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:58:06.367842 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 697/1401] 10.222.0.40 () {60 vars in 1384 bytes} [Fri Aug 26 10:58:06 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 26 10:58:06.431464 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-50640a74-0e29-4035-b9d9-1738f88d49fc None None] MFA Rules not processed for user `ca155a48e2024c32af3fd2e19e5fac11`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 10:58:06.432699 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-50640a74-0e29-4035-b9d9-1738f88d49fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 10:58:06.494718 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:58:06.542023 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 705/1402] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 26 10:58:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 10:58:06.849061 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67312813-274a-4b9d-ae1f-56cd348d1e1d service swift] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:58:06.850291 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-67312813-274a-4b9d-ae1f-56cd348d1e1d service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 10:58:06.948979 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67312813-274a-4b9d-ae1f-56cd348d1e1d service swift] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:58:07.060684 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:58:07.076925 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27fc333-1fb3-4fbc-8571-d25eb67785d7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f4b4a662db049ecacc865ecff10bdb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e30ccf0c1204727b786876c141dbcd2', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:58:07.077627 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27fc333-1fb3-4fbc-8571-d25eb67785d7 service nova] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:58:07.078011 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27fc333-1fb3-4fbc-8571-d25eb67785d7 service nova] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:58:07.078343 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27fc333-1fb3-4fbc-8571-d25eb67785d7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:58:07.080031 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-f27fc333-1fb3-4fbc-8571-d25eb67785d7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 10:58:07.186906 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f27fc333-1fb3-4fbc-8571-d25eb67785d7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:58:07.189454 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f27fc333-1fb3-4fbc-8571-d25eb67785d7 service nova] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:58:07.212747 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 698/1403] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 10:58:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 363 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 10:58:09.198117 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e2cf8b-bef9-40c1-a38f-8ab94da9b6f7 None None] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:58:09.198682 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e2cf8b-bef9-40c1-a38f-8ab94da9b6f7 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:58:09.198682 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e2cf8b-bef9-40c1-a38f-8ab94da9b6f7 None None] PATH_INFO: `/` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:58:09.201934 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 706/1404] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 26 10:58:09 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 10:58:09.209711 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c038408-dca2-4387-8877-3a7e7ff959ff None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:58:09.210303 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c038408-dca2-4387-8877-3a7e7ff959ff None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:58:09.210303 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c038408-dca2-4387-8877-3a7e7ff959ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:58:09.284934 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-8c038408-dca2-4387-8877-3a7e7ff959ff None None] MFA Rules not processed for user `ca155a48e2024c32af3fd2e19e5fac11`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 10:58:09.286205 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-8c038408-dca2-4387-8877-3a7e7ff959ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 10:58:09.355889 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:58:09.405416 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 699/1405] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 26 10:58:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 10:58:09.706803 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e2cf8b-bef9-40c1-a38f-8ab94da9b6f7 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:58:09.708464 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-49e2cf8b-bef9-40c1-a38f-8ab94da9b6f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 10:58:09.894711 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e2cf8b-bef9-40c1-a38f-8ab94da9b6f7 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:58:09.944527 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:58:10.033825 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5c5042-08ee-4a74-973b-b6cdb027a999 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f4b4a662db049ecacc865ecff10bdb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e30ccf0c1204727b786876c141dbcd2', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:58:10.034470 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5c5042-08ee-4a74-973b-b6cdb027a999 service nova] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:58:10.034748 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5c5042-08ee-4a74-973b-b6cdb027a999 service nova] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:58:10.035017 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5c5042-08ee-4a74-973b-b6cdb027a999 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:58:10.036659 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-ba5c5042-08ee-4a74-973b-b6cdb027a999 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 10:58:10.145034 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5c5042-08ee-4a74-973b-b6cdb027a999 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:58:10.146950 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5c5042-08ee-4a74-973b-b6cdb027a999 service nova] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:58:10.168050 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 707/1406] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 10:58:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 461 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 10:58:12.159697 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a33630-c5aa-4c7b-9b54-5739ddac5803 None None] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:58:12.160211 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a33630-c5aa-4c7b-9b54-5739ddac5803 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:58:12.160211 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a33630-c5aa-4c7b-9b54-5739ddac5803 None None] PATH_INFO: `/` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:58:12.161660 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 700/1407] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 26 10:58:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 10:58:12.167699 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780dfe94-f346-4da1-8a20-fd29f55121f0 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:58:12.168812 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780dfe94-f346-4da1-8a20-fd29f55121f0 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:58:12.168812 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780dfe94-f346-4da1-8a20-fd29f55121f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:58:12.240282 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:58:12.248488 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-780dfe94-f346-4da1-8a20-fd29f55121f0 None None] MFA Rules not processed for user `ca155a48e2024c32af3fd2e19e5fac11`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 10:58:12.249753 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-780dfe94-f346-4da1-8a20-fd29f55121f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 10:58:12.358877 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 708/1408] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 26 10:58:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 10:58:12.650175 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a33630-c5aa-4c7b-9b54-5739ddac5803 None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:58:12.652008 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-b4a33630-c5aa-4c7b-9b54-5739ddac5803 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 10:58:12.822303 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a33630-c5aa-4c7b-9b54-5739ddac5803 None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:58:12.883445 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:58:12.959794 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa3e4a0-18d4-4702-ac4f-8ce8df29612e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f4b4a662db049ecacc865ecff10bdb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e30ccf0c1204727b786876c141dbcd2', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:58:12.960308 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa3e4a0-18d4-4702-ac4f-8ce8df29612e service nova] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:58:12.960607 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa3e4a0-18d4-4702-ac4f-8ce8df29612e service nova] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:58:12.960871 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa3e4a0-18d4-4702-ac4f-8ce8df29612e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:58:12.962503 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-bfa3e4a0-18d4-4702-ac4f-8ce8df29612e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 10:58:13.073904 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa3e4a0-18d4-4702-ac4f-8ce8df29612e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:58:13.076429 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa3e4a0-18d4-4702-ac4f-8ce8df29612e service nova] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:58:13.096498 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 701/1409] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 10:58:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 447 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 10:58:22.356906 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=44 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:58:23.094263 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=77 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:58:28.397449 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1b78c7-bcfc-4167-a361-890d18ae5e78 None None] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:58:28.398028 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1b78c7-bcfc-4167-a361-890d18ae5e78 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:58:28.398028 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1b78c7-bcfc-4167-a361-890d18ae5e78 None None] PATH_INFO: `/` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:58:28.399351 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 709/1410] 10.222.0.40 () {54 vars in 918 bytes} [Fri Aug 26 10:58:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 10:58:28.404697 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abef2f6-2d3f-492f-9a43-eed16bb29ee0 None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:58:28.405154 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abef2f6-2d3f-492f-9a43-eed16bb29ee0 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:58:28.405275 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abef2f6-2d3f-492f-9a43-eed16bb29ee0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:58:28.479145 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-1abef2f6-2d3f-492f-9a43-eed16bb29ee0 None None] MFA Rules not processed for user `f29c5dfed31f42a9b5cbc0cd72e9fdc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 10:58:28.480998 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-1abef2f6-2d3f-492f-9a43-eed16bb29ee0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 10:58:28.593501 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 702/1411] 10.222.0.40 () {58 vars in 1015 bytes} [Fri Aug 26 10:58:28 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 10:58:28.608592 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5869b4b-8033-474e-a2f0-421fb6f0ac3c None None] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:58:28.608897 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5869b4b-8033-474e-a2f0-421fb6f0ac3c None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:58:28.609173 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5869b4b-8033-474e-a2f0-421fb6f0ac3c None None] PATH_INFO: `/` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:58:28.610542 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 710/1412] 10.222.0.40 () {54 vars in 918 bytes} [Fri Aug 26 10:58:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 10:58:28.615417 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e047f3d-adb1-4875-a0ed-00ff9534a8ae None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:58:28.615703 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e047f3d-adb1-4875-a0ed-00ff9534a8ae None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:58:28.615977 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e047f3d-adb1-4875-a0ed-00ff9534a8ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:58:28.688631 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:58:28.691388 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-8e047f3d-adb1-4875-a0ed-00ff9534a8ae None None] MFA Rules not processed for user `1f4b4a662db049ecacc865ecff10bdb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 10:58:28.693133 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-8e047f3d-adb1-4875-a0ed-00ff9534a8ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 10:58:28.807242 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 703/1413] 10.222.0.40 () {58 vars in 1015 bytes} [Fri Aug 26 10:58:28 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 10:58:31.252119 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddaa8f1-da99-48d9-96c6-315b3fd52d7b None None] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:58:31.252691 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddaa8f1-da99-48d9-96c6-315b3fd52d7b None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:58:31.252691 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddaa8f1-da99-48d9-96c6-315b3fd52d7b None None] PATH_INFO: `/` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:58:31.254211 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 711/1414] 127.0.0.1 () {54 vars in 917 bytes} [Fri Aug 26 10:58:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 10:58:31.259614 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d39f48b-83b2-4a85-99fb-d07f50685534 None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:58:31.260021 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d39f48b-83b2-4a85-99fb-d07f50685534 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:58:31.260155 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d39f48b-83b2-4a85-99fb-d07f50685534 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:58:31.329735 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-2d39f48b-83b2-4a85-99fb-d07f50685534 None None] MFA Rules not processed for user `ca155a48e2024c32af3fd2e19e5fac11`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 10:58:31.330976 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-2d39f48b-83b2-4a85-99fb-d07f50685534 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 10:58:31.387332 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:58:31.428658 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 704/1415] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Aug 26 10:58:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 10:58:31.717109 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ddaa8f1-da99-48d9-96c6-315b3fd52d7b None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:58:31.718940 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-3ddaa8f1-da99-48d9-96c6-315b3fd52d7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 10:58:31.863176 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ddaa8f1-da99-48d9-96c6-315b3fd52d7b None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:58:31.935574 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:58:31.982729 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6408468-6e14-4fa4-ac18-24ada00b32e1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f4b4a662db049ecacc865ecff10bdb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e30ccf0c1204727b786876c141dbcd2', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:58:31.983267 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6408468-6e14-4fa4-ac18-24ada00b32e1 service nova] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:58:31.983538 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6408468-6e14-4fa4-ac18-24ada00b32e1 service nova] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:58:31.983802 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6408468-6e14-4fa4-ac18-24ada00b32e1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:58:31.985388 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-a6408468-6e14-4fa4-ac18-24ada00b32e1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 10:58:32.073174 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6408468-6e14-4fa4-ac18-24ada00b32e1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:58:32.075346 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6408468-6e14-4fa4-ac18-24ada00b32e1 service nova] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:58:32.093832 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 712/1416] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 10:58:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 10:58:32.982297 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8217555-fc88-41fb-883b-9a1076b47cf9 None None] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:58:32.982862 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8217555-fc88-41fb-883b-9a1076b47cf9 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:58:32.982862 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8217555-fc88-41fb-883b-9a1076b47cf9 None None] PATH_INFO: `/` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:58:32.984182 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 705/1417] 127.0.0.1 () {54 vars in 901 bytes} [Fri Aug 26 10:58:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 10:58:32.989569 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c197362-43f7-4954-b072-ed06aacafe6b None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:58:32.990035 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c197362-43f7-4954-b072-ed06aacafe6b None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:58:32.990147 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c197362-43f7-4954-b072-ed06aacafe6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:58:33.065874 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-8c197362-43f7-4954-b072-ed06aacafe6b None None] MFA Rules not processed for user `ca155a48e2024c32af3fd2e19e5fac11`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 10:58:33.067075 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-8c197362-43f7-4954-b072-ed06aacafe6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 10:58:33.094067 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:58:33.162939 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 713/1418] 127.0.0.1 () {58 vars in 998 bytes} [Fri Aug 26 10:58:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 10:58:33.313261 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8217555-fc88-41fb-883b-9a1076b47cf9 None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:58:33.314828 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-e8217555-fc88-41fb-883b-9a1076b47cf9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 10:58:33.460863 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8217555-fc88-41fb-883b-9a1076b47cf9 None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:58:33.500792 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:58:33.573155 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4578323-b9db-49b9-98d3-2637d86197ee service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f4b4a662db049ecacc865ecff10bdb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e30ccf0c1204727b786876c141dbcd2', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:58:33.573747 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4578323-b9db-49b9-98d3-2637d86197ee service nova] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:58:33.574041 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4578323-b9db-49b9-98d3-2637d86197ee service nova] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:58:33.574311 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4578323-b9db-49b9-98d3-2637d86197ee service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:58:33.576586 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-d4578323-b9db-49b9-98d3-2637d86197ee service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 10:58:33.663641 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4578323-b9db-49b9-98d3-2637d86197ee service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:58:33.665209 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4578323-b9db-49b9-98d3-2637d86197ee service nova] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:58:33.683132 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 706/1419] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 10:58:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 10:58:34.574616 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4598761-a62d-447b-bcc5-f957fa892e43 None None] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:58:34.575186 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4598761-a62d-447b-bcc5-f957fa892e43 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:58:34.575186 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4598761-a62d-447b-bcc5-f957fa892e43 None None] PATH_INFO: `/` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:58:34.577387 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 714/1420] 127.0.0.1 () {54 vars in 901 bytes} [Fri Aug 26 10:58:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 10:58:34.582295 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ccf2067-2353-4198-85ad-8c6b7384247e None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:58:34.582738 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ccf2067-2353-4198-85ad-8c6b7384247e None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:58:34.582847 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ccf2067-2353-4198-85ad-8c6b7384247e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:58:34.636235 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:58:34.657930 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-1ccf2067-2353-4198-85ad-8c6b7384247e None None] MFA Rules not processed for user `ca155a48e2024c32af3fd2e19e5fac11`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 10:58:34.659147 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-1ccf2067-2353-4198-85ad-8c6b7384247e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 10:58:34.753507 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 707/1421] 127.0.0.1 () {58 vars in 998 bytes} [Fri Aug 26 10:58:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 10:58:34.906041 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4598761-a62d-447b-bcc5-f957fa892e43 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:58:34.907654 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-c4598761-a62d-447b-bcc5-f957fa892e43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 10:58:35.050713 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4598761-a62d-447b-bcc5-f957fa892e43 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:58:35.063025 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:58:35.162883 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ea4525-c01c-438a-bdb7-d58711bf2c36 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f4b4a662db049ecacc865ecff10bdb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e30ccf0c1204727b786876c141dbcd2', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:58:35.163501 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ea4525-c01c-438a-bdb7-d58711bf2c36 service nova] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:58:35.163777 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ea4525-c01c-438a-bdb7-d58711bf2c36 service nova] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:58:35.164044 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ea4525-c01c-438a-bdb7-d58711bf2c36 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:58:35.165644 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-97ea4525-c01c-438a-bdb7-d58711bf2c36 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 10:58:35.255750 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ea4525-c01c-438a-bdb7-d58711bf2c36 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:58:35.257330 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ea4525-c01c-438a-bdb7-d58711bf2c36 service nova] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:58:35.274831 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 715/1422] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 10:58:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 10:58:36.128904 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1f3ee9-5761-489c-916e-6207bcc58034 None None] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:58:36.129458 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1f3ee9-5761-489c-916e-6207bcc58034 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:58:36.129458 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1f3ee9-5761-489c-916e-6207bcc58034 None None] PATH_INFO: `/` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:58:36.130798 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 708/1423] 127.0.0.1 () {54 vars in 901 bytes} [Fri Aug 26 10:58:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 10:58:36.135955 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532e50d5-a8ad-4e19-9be0-ffac5e4e065b None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:58:36.136367 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532e50d5-a8ad-4e19-9be0-ffac5e4e065b None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:58:36.136502 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532e50d5-a8ad-4e19-9be0-ffac5e4e065b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:58:36.156765 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:58:36.212026 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-532e50d5-a8ad-4e19-9be0-ffac5e4e065b None None] MFA Rules not processed for user `ca155a48e2024c32af3fd2e19e5fac11`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 10:58:36.213233 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-532e50d5-a8ad-4e19-9be0-ffac5e4e065b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 10:58:36.308230 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 716/1424] 127.0.0.1 () {58 vars in 998 bytes} [Fri Aug 26 10:58:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 10:58:36.458100 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1f3ee9-5761-489c-916e-6207bcc58034 None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:58:36.459540 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-fb1f3ee9-5761-489c-916e-6207bcc58034 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 10:58:36.570614 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:58:36.608233 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1f3ee9-5761-489c-916e-6207bcc58034 None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:58:36.720992 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2bdeca-95bc-42a8-8dd4-d75d98be33a6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f4b4a662db049ecacc865ecff10bdb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e30ccf0c1204727b786876c141dbcd2', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:58:36.721512 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2bdeca-95bc-42a8-8dd4-d75d98be33a6 service nova] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:58:36.721787 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2bdeca-95bc-42a8-8dd4-d75d98be33a6 service nova] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:58:36.722082 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2bdeca-95bc-42a8-8dd4-d75d98be33a6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:58:36.723684 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-3c2bdeca-95bc-42a8-8dd4-d75d98be33a6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 10:58:36.801113 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:58:36.815090 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c2bdeca-95bc-42a8-8dd4-d75d98be33a6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:58:36.816685 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c2bdeca-95bc-42a8-8dd4-d75d98be33a6 service nova] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:58:36.834480 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 709/1425] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 10:58:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 10:58:37.709195 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9355953-7010-47b6-a5df-dcc65946d9af None None] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:58:37.709878 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9355953-7010-47b6-a5df-dcc65946d9af None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:58:37.709878 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9355953-7010-47b6-a5df-dcc65946d9af None None] PATH_INFO: `/` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:58:37.711168 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 717/1426] 127.0.0.1 () {54 vars in 901 bytes} [Fri Aug 26 10:58:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 10:58:37.716220 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f97686-a8f5-44d5-9969-00113ed01e7e None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:58:37.716688 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f97686-a8f5-44d5-9969-00113ed01e7e None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:58:37.717120 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f97686-a8f5-44d5-9969-00113ed01e7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:58:37.792746 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-f8f97686-a8f5-44d5-9969-00113ed01e7e None None] MFA Rules not processed for user `ca155a48e2024c32af3fd2e19e5fac11`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 10:58:37.793956 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-f8f97686-a8f5-44d5-9969-00113ed01e7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 10:58:37.890914 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 710/1427] 127.0.0.1 () {58 vars in 998 bytes} [Fri Aug 26 10:58:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 10:58:38.045804 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9355953-7010-47b6-a5df-dcc65946d9af None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:58:38.047294 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-e9355953-7010-47b6-a5df-dcc65946d9af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 10:58:38.130705 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:58:38.193011 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9355953-7010-47b6-a5df-dcc65946d9af None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:58:38.303773 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c8b1b5-7663-44ec-a90f-870274a19903 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f4b4a662db049ecacc865ecff10bdb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e30ccf0c1204727b786876c141dbcd2', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:58:38.304297 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c8b1b5-7663-44ec-a90f-870274a19903 service nova] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:58:38.304573 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c8b1b5-7663-44ec-a90f-870274a19903 service nova] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:58:38.304840 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c8b1b5-7663-44ec-a90f-870274a19903 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:58:38.306747 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-b8c8b1b5-7663-44ec-a90f-870274a19903 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 10:58:38.358310 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:58:38.400397 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c8b1b5-7663-44ec-a90f-870274a19903 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:58:38.401997 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c8b1b5-7663-44ec-a90f-870274a19903 service nova] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:58:38.421118 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 718/1428] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 10:58:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 10:58:39.233155 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0516b2b1-f36b-42e8-9387-eb175d81b61b None None] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:58:39.233736 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0516b2b1-f36b-42e8-9387-eb175d81b61b None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:58:39.233736 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0516b2b1-f36b-42e8-9387-eb175d81b61b None None] PATH_INFO: `/` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:58:39.235168 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 711/1429] 127.0.0.1 () {54 vars in 901 bytes} [Fri Aug 26 10:58:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 10:58:39.240248 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee8940b-40f9-4d25-a2a4-2b50011cff5f None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:58:39.240672 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee8940b-40f9-4d25-a2a4-2b50011cff5f None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:58:39.240813 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee8940b-40f9-4d25-a2a4-2b50011cff5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:58:39.322536 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-dee8940b-40f9-4d25-a2a4-2b50011cff5f None None] MFA Rules not processed for user `ca155a48e2024c32af3fd2e19e5fac11`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 10:58:39.323741 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-dee8940b-40f9-4d25-a2a4-2b50011cff5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 10:58:39.418555 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 719/1430] 127.0.0.1 () {58 vars in 998 bytes} [Fri Aug 26 10:58:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 10:58:39.569203 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0516b2b1-f36b-42e8-9387-eb175d81b61b None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:58:39.570827 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-0516b2b1-f36b-42e8-9387-eb175d81b61b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 10:58:39.620397 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:58:39.724931 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0516b2b1-f36b-42e8-9387-eb175d81b61b None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:58:39.838930 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6175da-e4b0-4af8-922f-05cae09d756f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f4b4a662db049ecacc865ecff10bdb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e30ccf0c1204727b786876c141dbcd2', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:58:39.839475 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6175da-e4b0-4af8-922f-05cae09d756f service nova] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:58:39.839749 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6175da-e4b0-4af8-922f-05cae09d756f service nova] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:58:39.840016 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6175da-e4b0-4af8-922f-05cae09d756f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:58:39.842165 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-2c6175da-e4b0-4af8-922f-05cae09d756f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 10:58:39.849170 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:58:39.933140 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c6175da-e4b0-4af8-922f-05cae09d756f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:58:39.934811 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c6175da-e4b0-4af8-922f-05cae09d756f service nova] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:58:39.953090 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 712/1431] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 10:58:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 384 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 10:58:40.771193 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ffa596-ee4c-477b-b04e-2dc4c312c223 None None] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:58:40.771765 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ffa596-ee4c-477b-b04e-2dc4c312c223 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:58:40.771765 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ffa596-ee4c-477b-b04e-2dc4c312c223 None None] PATH_INFO: `/` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:58:40.773102 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 720/1432] 127.0.0.1 () {54 vars in 901 bytes} [Fri Aug 26 10:58:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 10:58:40.778117 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f955cd-de41-40ad-ab33-b6a1f85024c9 None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:58:40.778527 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f955cd-de41-40ad-ab33-b6a1f85024c9 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:58:40.779017 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f955cd-de41-40ad-ab33-b6a1f85024c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:58:40.854715 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-a0f955cd-de41-40ad-ab33-b6a1f85024c9 None None] MFA Rules not processed for user `ca155a48e2024c32af3fd2e19e5fac11`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 10:58:40.855930 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-a0f955cd-de41-40ad-ab33-b6a1f85024c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 10:58:40.924689 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:58:40.954251 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 713/1433] 127.0.0.1 () {58 vars in 998 bytes} [Fri Aug 26 10:58:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 10:58:41.105386 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ffa596-ee4c-477b-b04e-2dc4c312c223 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:58:41.107450 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-b2ffa596-ee4c-477b-b04e-2dc4c312c223 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 10:58:41.125572 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:58:41.251264 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ffa596-ee4c-477b-b04e-2dc4c312c223 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:58:41.339670 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:58:41.367052 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a94fec-c5a7-4479-9923-61b7f563d17e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f4b4a662db049ecacc865ecff10bdb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e30ccf0c1204727b786876c141dbcd2', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:58:41.367591 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a94fec-c5a7-4479-9923-61b7f563d17e service nova] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:58:41.367866 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a94fec-c5a7-4479-9923-61b7f563d17e service nova] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:58:41.368132 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a94fec-c5a7-4479-9923-61b7f563d17e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:58:41.369727 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-e9a94fec-c5a7-4479-9923-61b7f563d17e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 10:58:41.457701 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a94fec-c5a7-4479-9923-61b7f563d17e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:58:41.459317 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a94fec-c5a7-4479-9923-61b7f563d17e service nova] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:58:41.478333 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 721/1434] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 10:58:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 10:58:42.277766 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781ca172-5e94-47a4-b793-68aff8b4b4b1 None None] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:58:42.278928 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781ca172-5e94-47a4-b793-68aff8b4b4b1 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:58:42.278928 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781ca172-5e94-47a4-b793-68aff8b4b4b1 None None] PATH_INFO: `/` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:58:42.280211 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 714/1435] 127.0.0.1 () {54 vars in 901 bytes} [Fri Aug 26 10:58:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 10:58:42.285307 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e09f2f-7897-498d-855a-5253224a35fb None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:58:42.285724 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e09f2f-7897-498d-855a-5253224a35fb None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:58:42.285833 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e09f2f-7897-498d-855a-5253224a35fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:58:42.364907 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-21e09f2f-7897-498d-855a-5253224a35fb None None] MFA Rules not processed for user `ca155a48e2024c32af3fd2e19e5fac11`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 10:58:42.366148 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-21e09f2f-7897-498d-855a-5253224a35fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 10:58:42.409332 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:58:42.464192 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 722/1436] 127.0.0.1 () {58 vars in 998 bytes} [Fri Aug 26 10:58:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 10:58:42.623562 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781ca172-5e94-47a4-b793-68aff8b4b4b1 None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:58:42.625201 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-781ca172-5e94-47a4-b793-68aff8b4b4b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 10:58:42.774274 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781ca172-5e94-47a4-b793-68aff8b4b4b1 None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:58:42.827626 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:58:42.887263 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc3a82d-aace-4af9-a84f-e19964c40a97 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f4b4a662db049ecacc865ecff10bdb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e30ccf0c1204727b786876c141dbcd2', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:58:42.888005 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc3a82d-aace-4af9-a84f-e19964c40a97 service nova] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:58:42.888282 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc3a82d-aace-4af9-a84f-e19964c40a97 service nova] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:58:42.888548 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc3a82d-aace-4af9-a84f-e19964c40a97 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:58:42.890181 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-2dc3a82d-aace-4af9-a84f-e19964c40a97 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 10:58:42.978103 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc3a82d-aace-4af9-a84f-e19964c40a97 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:58:42.979667 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc3a82d-aace-4af9-a84f-e19964c40a97 service nova] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:58:42.998217 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 715/1437] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 10:58:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 26 10:58:43.801095 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ed2cc0-858d-43d6-9b18-88bf63424dc2 None None] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:58:43.801670 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ed2cc0-858d-43d6-9b18-88bf63424dc2 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:58:43.801670 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ed2cc0-858d-43d6-9b18-88bf63424dc2 None None] PATH_INFO: `/` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:58:43.803062 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 723/1438] 127.0.0.1 () {54 vars in 901 bytes} [Fri Aug 26 10:58:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 10:58:43.807958 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6390dfd8-8fc5-47f1-82f0-a2cf73ab4772 None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:58:43.808383 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6390dfd8-8fc5-47f1-82f0-a2cf73ab4772 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:58:43.808539 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6390dfd8-8fc5-47f1-82f0-a2cf73ab4772 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:58:43.885175 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-6390dfd8-8fc5-47f1-82f0-a2cf73ab4772 None None] MFA Rules not processed for user `ca155a48e2024c32af3fd2e19e5fac11`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 10:58:43.886432 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-6390dfd8-8fc5-47f1-82f0-a2cf73ab4772 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 10:58:43.889202 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:58:43.989429 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 716/1439] 127.0.0.1 () {58 vars in 998 bytes} [Fri Aug 26 10:58:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 10:58:44.139782 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ed2cc0-858d-43d6-9b18-88bf63424dc2 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:58:44.141303 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-15ed2cc0-858d-43d6-9b18-88bf63424dc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 10:58:44.290566 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ed2cc0-858d-43d6-9b18-88bf63424dc2 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:58:44.317826 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:58:44.404039 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b633e7-3914-4cad-9a9d-d1851efe04cf service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f4b4a662db049ecacc865ecff10bdb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e30ccf0c1204727b786876c141dbcd2', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:58:44.404547 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b633e7-3914-4cad-9a9d-d1851efe04cf service nova] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:58:44.404826 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b633e7-3914-4cad-9a9d-d1851efe04cf service nova] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:58:44.405105 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b633e7-3914-4cad-9a9d-d1851efe04cf service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:58:44.406747 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-68b633e7-3914-4cad-9a9d-d1851efe04cf service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 10:58:44.495914 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b633e7-3914-4cad-9a9d-d1851efe04cf service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:58:44.498090 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b633e7-3914-4cad-9a9d-d1851efe04cf service nova] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:58:44.516748 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 724/1440] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 10:58:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 10:58:46.354073 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13478529-6e1d-43c4-92cd-f816f7d6caa9 None None] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:58:46.354602 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13478529-6e1d-43c4-92cd-f816f7d6caa9 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:58:46.354602 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13478529-6e1d-43c4-92cd-f816f7d6caa9 None None] PATH_INFO: `/` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:58:46.355912 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 717/1441] 127.0.0.1 () {54 vars in 917 bytes} [Fri Aug 26 10:58:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 10:58:46.361896 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ef50fc-0c53-42c8-843c-4f8bdd4ead22 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:58:46.362361 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ef50fc-0c53-42c8-843c-4f8bdd4ead22 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:58:46.362844 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ef50fc-0c53-42c8-843c-4f8bdd4ead22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:58:46.404497 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:58:46.430841 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-06ef50fc-0c53-42c8-843c-4f8bdd4ead22 None None] MFA Rules not processed for user `ca155a48e2024c32af3fd2e19e5fac11`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 10:58:46.432038 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-06ef50fc-0c53-42c8-843c-4f8bdd4ead22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 10:58:46.527058 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 725/1442] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Aug 26 10:58:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 10:58:46.812905 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13478529-6e1d-43c4-92cd-f816f7d6caa9 None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:58:46.814503 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-13478529-6e1d-43c4-92cd-f816f7d6caa9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 10:58:46.919786 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13478529-6e1d-43c4-92cd-f816f7d6caa9 None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:58:46.952112 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:58:46.993933 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a5a13f-9258-4601-9a7f-2845079d5afe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91da2acd2534423091537e1ba068d2b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e30ccf0c1204727b786876c141dbcd2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:58:46.994556 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a5a13f-9258-4601-9a7f-2845079d5afe service neutron] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:58:46.994829 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a5a13f-9258-4601-9a7f-2845079d5afe service neutron] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:58:46.995093 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a5a13f-9258-4601-9a7f-2845079d5afe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:58:46.996678 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-e1a5a13f-9258-4601-9a7f-2845079d5afe service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 10:58:47.104982 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a5a13f-9258-4601-9a7f-2845079d5afe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:58:47.106419 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a5a13f-9258-4601-9a7f-2845079d5afe service neutron] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:58:47.124844 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 718/1443] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 10:58:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 10:58:49.171487 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7129a973-f428-4bee-9aa9-1a84aeb01f7f None None] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:58:49.172015 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7129a973-f428-4bee-9aa9-1a84aeb01f7f None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:58:49.172015 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7129a973-f428-4bee-9aa9-1a84aeb01f7f None None] PATH_INFO: `/` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:58:49.173423 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 726/1444] 127.0.0.1 () {54 vars in 917 bytes} [Fri Aug 26 10:58:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 10:58:49.178554 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57894108-561d-45a4-9a4d-6fb88c0a2564 None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:58:49.178970 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57894108-561d-45a4-9a4d-6fb88c0a2564 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:58:49.179104 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57894108-561d-45a4-9a4d-6fb88c0a2564 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:58:49.245021 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-57894108-561d-45a4-9a4d-6fb88c0a2564 None None] MFA Rules not processed for user `ca155a48e2024c32af3fd2e19e5fac11`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 10:58:49.246339 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-57894108-561d-45a4-9a4d-6fb88c0a2564 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 10:58:49.260253 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:58:49.344798 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 719/1445] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Aug 26 10:58:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 10:58:49.633941 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7129a973-f428-4bee-9aa9-1a84aeb01f7f None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:58:49.635494 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-7129a973-f428-4bee-9aa9-1a84aeb01f7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 10:58:49.736939 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7129a973-f428-4bee-9aa9-1a84aeb01f7f None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:58:49.749070 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:58:49.813363 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25317ef-a4ec-47c0-95c5-4007b718ed27 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91da2acd2534423091537e1ba068d2b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e30ccf0c1204727b786876c141dbcd2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:58:49.813965 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25317ef-a4ec-47c0-95c5-4007b718ed27 service neutron] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:58:49.814264 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25317ef-a4ec-47c0-95c5-4007b718ed27 service neutron] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:58:49.814528 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25317ef-a4ec-47c0-95c5-4007b718ed27 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:58:49.816115 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-c25317ef-a4ec-47c0-95c5-4007b718ed27 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 10:58:49.943366 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c25317ef-a4ec-47c0-95c5-4007b718ed27 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:58:49.944821 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c25317ef-a4ec-47c0-95c5-4007b718ed27 service neutron] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:58:49.975137 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 727/1446] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 10:58:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 10:58:52.311008 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fe9be3-7c19-42b7-af02-ab31419bec5d None None] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:58:52.311538 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fe9be3-7c19-42b7-af02-ab31419bec5d None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:58:52.311538 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fe9be3-7c19-42b7-af02-ab31419bec5d None None] PATH_INFO: `/` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:58:52.312823 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 720/1447] 127.0.0.1 () {54 vars in 917 bytes} [Fri Aug 26 10:58:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 10:58:52.318153 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa88d3a3-51f3-46c6-948f-3da7a077228d None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:58:52.318531 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa88d3a3-51f3-46c6-948f-3da7a077228d None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:58:52.318699 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa88d3a3-51f3-46c6-948f-3da7a077228d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:58:52.367357 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:58:52.385168 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-fa88d3a3-51f3-46c6-948f-3da7a077228d None None] MFA Rules not processed for user `ca155a48e2024c32af3fd2e19e5fac11`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 10:58:52.386413 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-fa88d3a3-51f3-46c6-948f-3da7a077228d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 10:58:52.482990 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 728/1448] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Aug 26 10:58:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 10:58:52.768035 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88fe9be3-7c19-42b7-af02-ab31419bec5d None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:58:52.769593 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-88fe9be3-7c19-42b7-af02-ab31419bec5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 10:58:52.875347 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88fe9be3-7c19-42b7-af02-ab31419bec5d None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:58:52.917289 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:58:52.951797 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa01a7a-e724-46d4-962f-dc3f9299e7b9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91da2acd2534423091537e1ba068d2b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e30ccf0c1204727b786876c141dbcd2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:58:52.952394 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa01a7a-e724-46d4-962f-dc3f9299e7b9 service neutron] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:58:52.952665 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa01a7a-e724-46d4-962f-dc3f9299e7b9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:58:52.952927 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa01a7a-e724-46d4-962f-dc3f9299e7b9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:58:52.954539 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-6aa01a7a-e724-46d4-962f-dc3f9299e7b9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 10:58:53.065011 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa01a7a-e724-46d4-962f-dc3f9299e7b9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:58:53.067108 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa01a7a-e724-46d4-962f-dc3f9299e7b9 service neutron] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:58:53.085837 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 721/1449] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 10:58:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 26 10:58:55.153506 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b631a34c-b903-4e96-885f-0a066e5f8fd4 None None] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:58:55.154062 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b631a34c-b903-4e96-885f-0a066e5f8fd4 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:58:55.154062 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b631a34c-b903-4e96-885f-0a066e5f8fd4 None None] PATH_INFO: `/` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:58:55.155354 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 729/1450] 127.0.0.1 () {54 vars in 917 bytes} [Fri Aug 26 10:58:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 10:58:55.160965 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5e669d-b216-47a2-b0a0-6f1fb4df4de2 None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:58:55.161502 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5e669d-b216-47a2-b0a0-6f1fb4df4de2 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:58:55.161607 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5e669d-b216-47a2-b0a0-6f1fb4df4de2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:58:55.227789 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-fb5e669d-b216-47a2-b0a0-6f1fb4df4de2 None None] MFA Rules not processed for user `ca155a48e2024c32af3fd2e19e5fac11`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 10:58:55.228992 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-fb5e669d-b216-47a2-b0a0-6f1fb4df4de2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 10:58:55.251574 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:58:55.328833 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 722/1451] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Aug 26 10:58:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 10:58:55.611584 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b631a34c-b903-4e96-885f-0a066e5f8fd4 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:58:55.613131 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-b631a34c-b903-4e96-885f-0a066e5f8fd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 10:58:55.716409 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b631a34c-b903-4e96-885f-0a066e5f8fd4 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:58:55.736199 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:58:55.790461 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be6df57-2821-4663-a835-3693da3bc4ac service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91da2acd2534423091537e1ba068d2b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e30ccf0c1204727b786876c141dbcd2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:58:55.791066 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be6df57-2821-4663-a835-3693da3bc4ac service neutron] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:58:55.791344 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be6df57-2821-4663-a835-3693da3bc4ac service neutron] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:58:55.791615 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be6df57-2821-4663-a835-3693da3bc4ac service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:58:55.793219 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-0be6df57-2821-4663-a835-3693da3bc4ac service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 10:58:55.901140 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be6df57-2821-4663-a835-3693da3bc4ac service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:58:55.902691 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be6df57-2821-4663-a835-3693da3bc4ac service neutron] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:58:55.921471 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 730/1452] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 10:58:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 10:58:57.905642 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31d6618-b315-40a0-a431-7108fcf81f5d None None] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:58:57.906148 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31d6618-b315-40a0-a431-7108fcf81f5d None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:58:57.906247 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31d6618-b315-40a0-a431-7108fcf81f5d None None] PATH_INFO: `/` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:58:57.907592 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 723/1453] 127.0.0.1 () {54 vars in 917 bytes} [Fri Aug 26 10:58:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 10:58:57.912983 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aaae4b9-ad38-452d-a2e1-0b034450d638 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:58:57.913405 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aaae4b9-ad38-452d-a2e1-0b034450d638 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:58:57.913557 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aaae4b9-ad38-452d-a2e1-0b034450d638 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:58:57.977608 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:58:57.980997 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-6aaae4b9-ad38-452d-a2e1-0b034450d638 None None] MFA Rules not processed for user `ca155a48e2024c32af3fd2e19e5fac11`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 10:58:57.982679 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-6aaae4b9-ad38-452d-a2e1-0b034450d638 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 10:58:58.082199 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 731/1454] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Aug 26 10:58:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 10:58:58.372516 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31d6618-b315-40a0-a431-7108fcf81f5d None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:58:58.374094 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-c31d6618-b315-40a0-a431-7108fcf81f5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 10:58:58.477188 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31d6618-b315-40a0-a431-7108fcf81f5d None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:58:58.528484 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:58:58.551812 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f295278-9693-4a67-80ef-68c42937d764 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91da2acd2534423091537e1ba068d2b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e30ccf0c1204727b786876c141dbcd2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:58:58.552438 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f295278-9693-4a67-80ef-68c42937d764 service neutron] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:58:58.552714 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f295278-9693-4a67-80ef-68c42937d764 service neutron] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:58:58.552985 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f295278-9693-4a67-80ef-68c42937d764 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:58:58.554602 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-5f295278-9693-4a67-80ef-68c42937d764 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 10:58:58.663672 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f295278-9693-4a67-80ef-68c42937d764 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:58:58.665106 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f295278-9693-4a67-80ef-68c42937d764 service neutron] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:58:58.682884 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 724/1455] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 10:58:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 26 10:59:00.711470 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afe2438-5df2-4f1d-b2d7-8e6d29fd114b None None] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:59:00.712094 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afe2438-5df2-4f1d-b2d7-8e6d29fd114b None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:59:00.712094 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afe2438-5df2-4f1d-b2d7-8e6d29fd114b None None] PATH_INFO: `/` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:59:00.713349 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 732/1456] 127.0.0.1 () {54 vars in 917 bytes} [Fri Aug 26 10:59:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 10:59:00.719494 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a580af64-d8f2-4ad9-9882-4f6713e1e8ee None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:59:00.719948 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a580af64-d8f2-4ad9-9882-4f6713e1e8ee None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:59:00.720046 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a580af64-d8f2-4ad9-9882-4f6713e1e8ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:59:00.787005 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-a580af64-d8f2-4ad9-9882-4f6713e1e8ee None None] MFA Rules not processed for user `ca155a48e2024c32af3fd2e19e5fac11`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 10:59:00.788211 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-a580af64-d8f2-4ad9-9882-4f6713e1e8ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 10:59:00.817273 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:59:00.884526 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 725/1457] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Aug 26 10:59:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 10:59:01.168892 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6afe2438-5df2-4f1d-b2d7-8e6d29fd114b None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:59:01.170967 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-6afe2438-5df2-4f1d-b2d7-8e6d29fd114b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 10:59:01.276419 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6afe2438-5df2-4f1d-b2d7-8e6d29fd114b None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:59:01.304729 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:59:01.350193 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed29752-a578-4504-b4a0-25d8aee93565 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91da2acd2534423091537e1ba068d2b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e30ccf0c1204727b786876c141dbcd2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:59:01.350793 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed29752-a578-4504-b4a0-25d8aee93565 service neutron] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:59:01.351069 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed29752-a578-4504-b4a0-25d8aee93565 service neutron] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:59:01.351344 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed29752-a578-4504-b4a0-25d8aee93565 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:59:01.352939 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-0ed29752-a578-4504-b4a0-25d8aee93565 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 10:59:01.461355 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed29752-a578-4504-b4a0-25d8aee93565 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:59:01.462809 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed29752-a578-4504-b4a0-25d8aee93565 service neutron] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:59:01.481926 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 733/1458] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 10:59:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 10:59:03.436867 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a207d7-f6f8-41fb-b2bb-e476c9ffa1b0 None None] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:59:03.437409 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a207d7-f6f8-41fb-b2bb-e476c9ffa1b0 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:59:03.437409 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a207d7-f6f8-41fb-b2bb-e476c9ffa1b0 None None] PATH_INFO: `/` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:59:03.438826 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 726/1459] 127.0.0.1 () {54 vars in 917 bytes} [Fri Aug 26 10:59:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 26 10:59:03.444561 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6adf0458-1995-4c79-9c05-7be3ffa49f46 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:59:03.444996 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6adf0458-1995-4c79-9c05-7be3ffa49f46 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:59:03.445111 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6adf0458-1995-4c79-9c05-7be3ffa49f46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:59:03.511622 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-6adf0458-1995-4c79-9c05-7be3ffa49f46 None None] MFA Rules not processed for user `ca155a48e2024c32af3fd2e19e5fac11`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 10:59:03.513357 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-6adf0458-1995-4c79-9c05-7be3ffa49f46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 10:59:03.521562 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:59:03.611668 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 734/1460] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Aug 26 10:59:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 10:59:03.893144 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a207d7-f6f8-41fb-b2bb-e476c9ffa1b0 None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:59:03.894709 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-b4a207d7-f6f8-41fb-b2bb-e476c9ffa1b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 10:59:03.995620 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a207d7-f6f8-41fb-b2bb-e476c9ffa1b0 None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:59:04.049922 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:59:04.068350 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c951ceda-5efd-4260-a8bd-b14b0e17149f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91da2acd2534423091537e1ba068d2b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e30ccf0c1204727b786876c141dbcd2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:59:04.068949 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c951ceda-5efd-4260-a8bd-b14b0e17149f service neutron] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:59:04.069224 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c951ceda-5efd-4260-a8bd-b14b0e17149f service neutron] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:59:04.069488 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c951ceda-5efd-4260-a8bd-b14b0e17149f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:59:04.071117 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-c951ceda-5efd-4260-a8bd-b14b0e17149f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 10:59:04.178983 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c951ceda-5efd-4260-a8bd-b14b0e17149f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:59:04.180320 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c951ceda-5efd-4260-a8bd-b14b0e17149f service neutron] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:59:04.199323 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 727/1461] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 10:59:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 10:59:06.163966 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8683c1a-7b03-49ef-bdb8-572a1c2f949b None None] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:59:06.164493 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8683c1a-7b03-49ef-bdb8-572a1c2f949b None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:59:06.164493 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8683c1a-7b03-49ef-bdb8-572a1c2f949b None None] PATH_INFO: `/` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:59:06.165780 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 735/1462] 127.0.0.1 () {54 vars in 917 bytes} [Fri Aug 26 10:59:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 10:59:06.171355 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc5a4a8-0a82-466e-a093-d0e26625a0c7 None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:59:06.171748 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc5a4a8-0a82-466e-a093-d0e26625a0c7 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:59:06.171932 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc5a4a8-0a82-466e-a093-d0e26625a0c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:59:06.237687 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-9bc5a4a8-0a82-466e-a093-d0e26625a0c7 None None] MFA Rules not processed for user `ca155a48e2024c32af3fd2e19e5fac11`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 10:59:06.239190 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-9bc5a4a8-0a82-466e-a093-d0e26625a0c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 10:59:06.278353 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:59:06.336131 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 728/1463] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Aug 26 10:59:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 10:59:06.619498 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8683c1a-7b03-49ef-bdb8-572a1c2f949b None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:59:06.621043 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-f8683c1a-7b03-49ef-bdb8-572a1c2f949b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 10:59:06.724147 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8683c1a-7b03-49ef-bdb8-572a1c2f949b None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:59:06.761714 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:59:06.798238 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6edcb74-0404-4c82-b015-28186fa7b857 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91da2acd2534423091537e1ba068d2b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e30ccf0c1204727b786876c141dbcd2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:59:06.798838 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6edcb74-0404-4c82-b015-28186fa7b857 service neutron] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:59:06.799117 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6edcb74-0404-4c82-b015-28186fa7b857 service neutron] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:59:06.799390 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6edcb74-0404-4c82-b015-28186fa7b857 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:59:06.800998 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-f6edcb74-0404-4c82-b015-28186fa7b857 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 10:59:06.909680 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6edcb74-0404-4c82-b015-28186fa7b857 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:59:06.911160 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6edcb74-0404-4c82-b015-28186fa7b857 service neutron] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:59:06.929728 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 736/1464] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 10:59:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 10:59:08.894857 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26d8dad-ec69-4dd4-b45e-d23ad37ae9f0 None None] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:59:08.895395 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26d8dad-ec69-4dd4-b45e-d23ad37ae9f0 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:59:08.895509 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26d8dad-ec69-4dd4-b45e-d23ad37ae9f0 None None] PATH_INFO: `/` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:59:08.896791 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 729/1465] 127.0.0.1 () {54 vars in 917 bytes} [Fri Aug 26 10:59:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 10:59:08.902092 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bec7e05-12db-49b9-8e8e-a5a21140c7b7 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:59:08.902514 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bec7e05-12db-49b9-8e8e-a5a21140c7b7 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:59:08.902663 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bec7e05-12db-49b9-8e8e-a5a21140c7b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:59:08.968210 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-6bec7e05-12db-49b9-8e8e-a5a21140c7b7 None None] MFA Rules not processed for user `ca155a48e2024c32af3fd2e19e5fac11`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 10:59:08.969442 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-6bec7e05-12db-49b9-8e8e-a5a21140c7b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 10:59:08.987736 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:59:09.067484 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 737/1466] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Aug 26 10:59:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 10:59:09.358189 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26d8dad-ec69-4dd4-b45e-d23ad37ae9f0 None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:59:09.359764 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-a26d8dad-ec69-4dd4-b45e-d23ad37ae9f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 10:59:09.464392 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26d8dad-ec69-4dd4-b45e-d23ad37ae9f0 None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:59:09.536284 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:59:09.543873 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9bd9436-494f-4913-9c9b-ae710e904b0d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91da2acd2534423091537e1ba068d2b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e30ccf0c1204727b786876c141dbcd2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:59:09.544479 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bd9436-494f-4913-9c9b-ae710e904b0d service neutron] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:59:09.544751 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bd9436-494f-4913-9c9b-ae710e904b0d service neutron] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:59:09.545014 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bd9436-494f-4913-9c9b-ae710e904b0d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:59:09.546634 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-c9bd9436-494f-4913-9c9b-ae710e904b0d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 10:59:09.656712 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9bd9436-494f-4913-9c9b-ae710e904b0d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:59:09.658175 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9bd9436-494f-4913-9c9b-ae710e904b0d service neutron] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:59:09.676391 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 730/1467] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 10:59:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 10:59:11.623490 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42407395-c0c1-493c-b6aa-b47edd922d2e None None] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:59:11.624063 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42407395-c0c1-493c-b6aa-b47edd922d2e None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:59:11.624063 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42407395-c0c1-493c-b6aa-b47edd922d2e None None] PATH_INFO: `/` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:59:11.625368 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 738/1468] 127.0.0.1 () {54 vars in 917 bytes} [Fri Aug 26 10:59:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 26 10:59:11.630712 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4341667-3c2c-4278-94cc-29fd32adc527 None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:59:11.631163 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4341667-3c2c-4278-94cc-29fd32adc527 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:59:11.631268 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4341667-3c2c-4278-94cc-29fd32adc527 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:59:11.697820 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-c4341667-3c2c-4278-94cc-29fd32adc527 None None] MFA Rules not processed for user `ca155a48e2024c32af3fd2e19e5fac11`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 10:59:11.699060 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-c4341667-3c2c-4278-94cc-29fd32adc527 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 10:59:11.747523 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:59:11.796980 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 731/1469] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Aug 26 10:59:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 10:59:12.080996 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42407395-c0c1-493c-b6aa-b47edd922d2e None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:59:12.082582 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-42407395-c0c1-493c-b6aa-b47edd922d2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 10:59:12.185095 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42407395-c0c1-493c-b6aa-b47edd922d2e None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:59:12.233319 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:59:12.260459 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca2bb51-03cb-4281-a9cd-3a5f4f632ea8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91da2acd2534423091537e1ba068d2b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e30ccf0c1204727b786876c141dbcd2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:59:12.261075 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca2bb51-03cb-4281-a9cd-3a5f4f632ea8 service neutron] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:59:12.261356 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca2bb51-03cb-4281-a9cd-3a5f4f632ea8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:59:12.261629 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca2bb51-03cb-4281-a9cd-3a5f4f632ea8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:59:12.263552 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-dca2bb51-03cb-4281-a9cd-3a5f4f632ea8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 10:59:12.371742 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dca2bb51-03cb-4281-a9cd-3a5f4f632ea8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:59:12.373198 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dca2bb51-03cb-4281-a9cd-3a5f4f632ea8 service neutron] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:59:12.391776 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 739/1470] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 10:59:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 10:59:21.795164 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=22 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:59:22.389478 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=66 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:59:31.896300 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062715aa-5078-4a63-8d8a-5da2cbd18ea8 None None] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:59:31.896738 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062715aa-5078-4a63-8d8a-5da2cbd18ea8 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:59:31.896864 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062715aa-5078-4a63-8d8a-5da2cbd18ea8 None None] PATH_INFO: `/` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:59:31.898239 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 732/1471] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 26 10:59:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 10:59:31.904214 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f078667f-78fe-4249-bd01-28fb075dce25 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:59:31.904658 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f078667f-78fe-4249-bd01-28fb075dce25 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:59:31.904768 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f078667f-78fe-4249-bd01-28fb075dce25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:59:31.969965 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-f078667f-78fe-4249-bd01-28fb075dce25 None None] MFA Rules not processed for user `ca155a48e2024c32af3fd2e19e5fac11`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 10:59:31.971255 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-f078667f-78fe-4249-bd01-28fb075dce25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 10:59:32.066816 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 740/1472] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 26 10:59:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 10:59:32.365397 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93aad19-1b07-400b-9318-42bf9700183c None None] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:59:32.365679 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93aad19-1b07-400b-9318-42bf9700183c None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:59:32.365950 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93aad19-1b07-400b-9318-42bf9700183c None None] PATH_INFO: `/` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:59:32.367330 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 733/1473] 10.222.0.40 () {54 vars in 967 bytes} [Fri Aug 26 10:59:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 10:59:32.372665 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c6eca1-26bb-46bc-918b-ba5582186122 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:59:32.372946 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c6eca1-26bb-46bc-918b-ba5582186122 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:59:32.373217 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c6eca1-26bb-46bc-918b-ba5582186122 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:59:32.447401 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-d1c6eca1-26bb-46bc-918b-ba5582186122 None None] MFA Rules not processed for user `8631d7ecfd2849daa8694aecccd22a05`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 10:59:32.449287 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-d1c6eca1-26bb-46bc-918b-ba5582186122 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 10:59:32.469335 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:59:32.525714 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 741/1474] 10.222.0.40 () {58 vars in 1064 bytes} [Fri Aug 26 10:59:32 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 10:59:32.533439 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93aad19-1b07-400b-9318-42bf9700183c None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:59:32.534970 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-d93aad19-1b07-400b-9318-42bf9700183c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 10:59:32.637161 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93aad19-1b07-400b-9318-42bf9700183c None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:59:32.707705 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c60488-7463-4c9a-9142-76758fcd0e2d service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8631d7ecfd2849daa8694aecccd22a05', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e30ccf0c1204727b786876c141dbcd2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:59:32.708294 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c60488-7463-4c9a-9142-76758fcd0e2d service glance] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:59:32.708566 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c60488-7463-4c9a-9142-76758fcd0e2d service glance] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:59:32.708830 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c60488-7463-4c9a-9142-76758fcd0e2d service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:59:32.710435 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-d7c60488-7463-4c9a-9142-76758fcd0e2d service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 10:59:32.763556 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:59:32.830346 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c60488-7463-4c9a-9142-76758fcd0e2d service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:59:32.831800 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c60488-7463-4c9a-9142-76758fcd0e2d service glance] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:59:32.850929 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 734/1475] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 10:59:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 10:59:34.853693 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef5e5d9-9a2a-492e-994b-12e4fca4a3cf None None] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:59:34.854227 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef5e5d9-9a2a-492e-994b-12e4fca4a3cf None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:59:34.854324 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef5e5d9-9a2a-492e-994b-12e4fca4a3cf None None] PATH_INFO: `/` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:59:34.856245 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 742/1476] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 26 10:59:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 10:59:34.861747 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964bafd4-6f18-4d7b-88cd-48a9535c7f67 None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:59:34.862210 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964bafd4-6f18-4d7b-88cd-48a9535c7f67 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:59:34.862335 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964bafd4-6f18-4d7b-88cd-48a9535c7f67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:59:34.930126 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-964bafd4-6f18-4d7b-88cd-48a9535c7f67 None None] MFA Rules not processed for user `ca155a48e2024c32af3fd2e19e5fac11`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 10:59:34.931330 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-964bafd4-6f18-4d7b-88cd-48a9535c7f67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 10:59:35.027850 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 735/1477] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 26 10:59:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 10:59:35.312001 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef5e5d9-9a2a-492e-994b-12e4fca4a3cf None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:59:35.313612 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-bef5e5d9-9a2a-492e-994b-12e4fca4a3cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 10:59:35.437239 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef5e5d9-9a2a-492e-994b-12e4fca4a3cf None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:59:35.484887 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:59:35.530769 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af60c1c-14ca-4bab-a648-b38cdaa1da66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:59:35.531376 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af60c1c-14ca-4bab-a648-b38cdaa1da66 admin admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:59:35.531647 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af60c1c-14ca-4bab-a648-b38cdaa1da66 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:59:35.531919 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af60c1c-14ca-4bab-a648-b38cdaa1da66 admin admin] PATH_INFO: `/v3/projects` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:59:35.532549 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af60c1c-14ca-4bab-a648-b38cdaa1da66 admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:59:35.534316 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: WARNING py.warnings [None req-3af60c1c-14ca-4bab-a648-b38cdaa1da66 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 26 10:59:35.534316 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: warnings.warn(msg) Aug 26 10:59:35.534316 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: Aug 26 10:59:35.535466 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af60c1c-14ca-4bab-a648-b38cdaa1da66 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:59:35.554204 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 743/1478] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Aug 26 10:59:35 2022] GET /identity/v3/projects => generated 2553 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 10:59:37.490707 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d5dd40-a62d-41b1-9d40-29cbb9815d49 None None] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:59:37.491302 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d5dd40-a62d-41b1-9d40-29cbb9815d49 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:59:37.491302 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d5dd40-a62d-41b1-9d40-29cbb9815d49 None None] PATH_INFO: `/` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:59:37.492613 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 736/1479] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 26 10:59:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 26 10:59:37.498419 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6141d672-1df3-43a9-8d03-63a2ab8195f9 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:59:37.498821 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6141d672-1df3-43a9-8d03-63a2ab8195f9 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:59:37.498969 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6141d672-1df3-43a9-8d03-63a2ab8195f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:59:37.566146 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-6141d672-1df3-43a9-8d03-63a2ab8195f9 None None] MFA Rules not processed for user `ca155a48e2024c32af3fd2e19e5fac11`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 10:59:37.567384 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-6141d672-1df3-43a9-8d03-63a2ab8195f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 10:59:37.662658 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 744/1480] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 26 10:59:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 10:59:37.958603 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d5dd40-a62d-41b1-9d40-29cbb9815d49 None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:59:37.960278 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-18d5dd40-a62d-41b1-9d40-29cbb9815d49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 10:59:37.970128 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:59:38.119304 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d5dd40-a62d-41b1-9d40-29cbb9815d49 None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:59:38.198477 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:59:38.232587 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfbe45da-ca72-45d4-9efc-08c6d5578d74 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f4b4a662db049ecacc865ecff10bdb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e30ccf0c1204727b786876c141dbcd2', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:59:38.233182 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbe45da-ca72-45d4-9efc-08c6d5578d74 service nova] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:59:38.233455 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbe45da-ca72-45d4-9efc-08c6d5578d74 service nova] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:59:38.233720 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbe45da-ca72-45d4-9efc-08c6d5578d74 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:59:38.235348 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-bfbe45da-ca72-45d4-9efc-08c6d5578d74 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 10:59:38.321792 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfbe45da-ca72-45d4-9efc-08c6d5578d74 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:59:38.323395 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfbe45da-ca72-45d4-9efc-08c6d5578d74 service nova] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:59:38.342126 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 737/1481] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 10:59:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 384 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 10:59:40.430329 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8035c4e-d4b1-4544-abcd-21f1e423144f None None] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:59:40.430933 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8035c4e-d4b1-4544-abcd-21f1e423144f None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:59:40.430933 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8035c4e-d4b1-4544-abcd-21f1e423144f None None] PATH_INFO: `/` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:59:40.432200 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 745/1482] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 26 10:59:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 10:59:40.437363 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873ffa90-f570-4585-9a23-7f97c4acb588 None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:59:40.437821 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873ffa90-f570-4585-9a23-7f97c4acb588 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:59:40.437936 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873ffa90-f570-4585-9a23-7f97c4acb588 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:59:40.503771 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-873ffa90-f570-4585-9a23-7f97c4acb588 None None] MFA Rules not processed for user `ca155a48e2024c32af3fd2e19e5fac11`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 10:59:40.504979 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-873ffa90-f570-4585-9a23-7f97c4acb588 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 10:59:40.552030 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:59:40.602527 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 738/1483] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 26 10:59:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 10:59:40.905206 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8035c4e-d4b1-4544-abcd-21f1e423144f None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:59:40.906792 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-f8035c4e-d4b1-4544-abcd-21f1e423144f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 10:59:40.931228 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:59:41.011847 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8035c4e-d4b1-4544-abcd-21f1e423144f None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:59:41.086711 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83fc97c8-9241-467e-a13b-db36111304c6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91da2acd2534423091537e1ba068d2b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e30ccf0c1204727b786876c141dbcd2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:59:41.087239 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fc97c8-9241-467e-a13b-db36111304c6 service neutron] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:59:41.087516 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fc97c8-9241-467e-a13b-db36111304c6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:59:41.087788 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fc97c8-9241-467e-a13b-db36111304c6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:59:41.089528 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-83fc97c8-9241-467e-a13b-db36111304c6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 10:59:41.164070 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:59:41.200014 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83fc97c8-9241-467e-a13b-db36111304c6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:59:41.201482 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83fc97c8-9241-467e-a13b-db36111304c6 service neutron] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:59:41.220556 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 746/1484] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 10:59:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 10:59:43.209770 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0d13f9-fc15-4583-9c36-4280ad3021e8 None None] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:59:43.210373 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0d13f9-fc15-4583-9c36-4280ad3021e8 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:59:43.210373 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0d13f9-fc15-4583-9c36-4280ad3021e8 None None] PATH_INFO: `/` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:59:43.211700 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 739/1485] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 26 10:59:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 10:59:43.217626 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cce9ffd-080a-4b6e-a4f0-2401ebfaa735 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:59:43.218063 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cce9ffd-080a-4b6e-a4f0-2401ebfaa735 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:59:43.218204 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cce9ffd-080a-4b6e-a4f0-2401ebfaa735 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:59:43.283804 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-7cce9ffd-080a-4b6e-a4f0-2401ebfaa735 None None] MFA Rules not processed for user `ca155a48e2024c32af3fd2e19e5fac11`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 10:59:43.285019 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-7cce9ffd-080a-4b6e-a4f0-2401ebfaa735 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 10:59:43.384989 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 747/1486] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 26 10:59:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 10:59:43.680212 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0d13f9-fc15-4583-9c36-4280ad3021e8 None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:59:43.681764 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-fe0d13f9-fc15-4583-9c36-4280ad3021e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 10:59:43.785212 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0d13f9-fc15-4583-9c36-4280ad3021e8 None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:59:43.858232 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:59:43.860217 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7086a6a7-dd11-4b06-b7d7-e3b48c9220af service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91da2acd2534423091537e1ba068d2b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e30ccf0c1204727b786876c141dbcd2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:59:43.860900 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7086a6a7-dd11-4b06-b7d7-e3b48c9220af service neutron] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:59:43.861350 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7086a6a7-dd11-4b06-b7d7-e3b48c9220af service neutron] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:59:43.861782 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7086a6a7-dd11-4b06-b7d7-e3b48c9220af service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:59:43.863595 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-7086a6a7-dd11-4b06-b7d7-e3b48c9220af service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 10:59:43.973601 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7086a6a7-dd11-4b06-b7d7-e3b48c9220af service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:59:43.974973 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7086a6a7-dd11-4b06-b7d7-e3b48c9220af service neutron] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:59:43.993904 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 740/1487] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 10:59:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 10:59:46.012064 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e792d8a-524d-44ce-866b-d1890add2927 None None] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:59:46.012648 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e792d8a-524d-44ce-866b-d1890add2927 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:59:46.012648 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e792d8a-524d-44ce-866b-d1890add2927 None None] PATH_INFO: `/` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:59:46.013940 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 748/1488] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 26 10:59:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 10:59:46.019141 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4ace72-08d3-4b3d-b6bf-92041e8b8de6 None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:59:46.019562 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4ace72-08d3-4b3d-b6bf-92041e8b8de6 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:59:46.019694 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4ace72-08d3-4b3d-b6bf-92041e8b8de6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:59:46.085529 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-8f4ace72-08d3-4b3d-b6bf-92041e8b8de6 None None] MFA Rules not processed for user `ca155a48e2024c32af3fd2e19e5fac11`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 10:59:46.086767 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-8f4ace72-08d3-4b3d-b6bf-92041e8b8de6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 10:59:46.140072 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:59:46.186788 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 741/1489] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 26 10:59:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 10:59:46.485190 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e792d8a-524d-44ce-866b-d1890add2927 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:59:46.486772 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-6e792d8a-524d-44ce-866b-d1890add2927 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 10:59:46.521548 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:59:46.588764 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e792d8a-524d-44ce-866b-d1890add2927 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:59:46.661797 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed26ec20-5c62-4a07-b5ca-b7c33dcfe0c0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91da2acd2534423091537e1ba068d2b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e30ccf0c1204727b786876c141dbcd2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:59:46.662538 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed26ec20-5c62-4a07-b5ca-b7c33dcfe0c0 service neutron] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:59:46.662817 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed26ec20-5c62-4a07-b5ca-b7c33dcfe0c0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:59:46.663091 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed26ec20-5c62-4a07-b5ca-b7c33dcfe0c0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:59:46.664705 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-ed26ec20-5c62-4a07-b5ca-b7c33dcfe0c0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 10:59:46.743825 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:59:46.772517 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed26ec20-5c62-4a07-b5ca-b7c33dcfe0c0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:59:46.773945 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed26ec20-5c62-4a07-b5ca-b7c33dcfe0c0 service neutron] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:59:46.793352 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 749/1490] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 10:59:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 10:59:48.997393 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891e1014-b86d-4ca3-976b-f5f4410ebc17 None None] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:59:48.997950 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891e1014-b86d-4ca3-976b-f5f4410ebc17 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:59:48.997950 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891e1014-b86d-4ca3-976b-f5f4410ebc17 None None] PATH_INFO: `/` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:59:48.999327 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 742/1491] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 26 10:59:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 10:59:49.004527 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f06cb1-3761-4cd8-b36c-29251a3d9404 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:59:49.004949 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f06cb1-3761-4cd8-b36c-29251a3d9404 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:59:49.005047 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f06cb1-3761-4cd8-b36c-29251a3d9404 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:59:49.070054 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-c5f06cb1-3761-4cd8-b36c-29251a3d9404 None None] MFA Rules not processed for user `ca155a48e2024c32af3fd2e19e5fac11`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 10:59:49.071201 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-c5f06cb1-3761-4cd8-b36c-29251a3d9404 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 10:59:49.167917 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 750/1492] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 26 10:59:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 10:59:49.456156 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891e1014-b86d-4ca3-976b-f5f4410ebc17 None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 10:59:49.457708 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-891e1014-b86d-4ca3-976b-f5f4410ebc17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 10:59:49.562951 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891e1014-b86d-4ca3-976b-f5f4410ebc17 None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 10:59:49.641149 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168defc1-8173-4aab-b543-6c351891434f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91da2acd2534423091537e1ba068d2b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e30ccf0c1204727b786876c141dbcd2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 10:59:49.641749 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168defc1-8173-4aab-b543-6c351891434f service neutron] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 10:59:49.642050 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168defc1-8173-4aab-b543-6c351891434f service neutron] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 10:59:49.642315 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168defc1-8173-4aab-b543-6c351891434f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 10:59:49.643900 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-168defc1-8173-4aab-b543-6c351891434f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 10:59:49.653332 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:59:49.754558 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-168defc1-8173-4aab-b543-6c351891434f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 10:59:49.755899 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-168defc1-8173-4aab-b543-6c351891434f service neutron] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 10:59:49.774373 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 743/1493] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 10:59:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 10:59:59.165932 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=82 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 10:59:59.772125 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=50 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 11:01:02.331093 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af60983-edf5-420e-9b76-d7626c5ea884 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:01:02.331648 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af60983-edf5-420e-9b76-d7626c5ea884 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:01:02.331648 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af60983-edf5-420e-9b76-d7626c5ea884 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:01:02.406828 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-7af60983-edf5-420e-9b76-d7626c5ea884 None None] MFA Rules not processed for user `ca155a48e2024c32af3fd2e19e5fac11`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 11:01:02.408061 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-7af60983-edf5-420e-9b76-d7626c5ea884 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:01:02.502371 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 751/1494] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 11:01:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 11:01:02.511144 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514cf887-8a74-4f5b-960c-916f09a4b36d None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:01:02.511646 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514cf887-8a74-4f5b-960c-916f09a4b36d None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:01:02.511757 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514cf887-8a74-4f5b-960c-916f09a4b36d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:01:02.586748 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-514cf887-8a74-4f5b-960c-916f09a4b36d None None] MFA Rules not processed for user `ca155a48e2024c32af3fd2e19e5fac11`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 11:01:02.588016 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-514cf887-8a74-4f5b-960c-916f09a4b36d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:01:02.682667 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 744/1495] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 11:01:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 11:01:02.687403 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af60983-edf5-420e-9b76-d7626c5ea884 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:01:02.688614 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-7af60983-edf5-420e-9b76-d7626c5ea884 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:01:02.700592 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af60983-edf5-420e-9b76-d7626c5ea884 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:01:02.760837 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 11:01:02.794508 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020f395a-18a5-4343-89a9-27db7c1b4de6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:01:02.795114 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020f395a-18a5-4343-89a9-27db7c1b4de6 admin admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:01:02.795391 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020f395a-18a5-4343-89a9-27db7c1b4de6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:01:02.795638 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020f395a-18a5-4343-89a9-27db7c1b4de6 admin admin] PATH_INFO: `/v3/domains` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:01:02.796263 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-020f395a-18a5-4343-89a9-27db7c1b4de6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 11:01:02.796560 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-020f395a-18a5-4343-89a9-27db7c1b4de6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:01:02.797558 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: WARNING py.warnings [None req-020f395a-18a5-4343-89a9-27db7c1b4de6 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 26 11:01:02.797558 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: warnings.warn(msg) Aug 26 11:01:02.797558 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: Aug 26 11:01:02.798459 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-020f395a-18a5-4343-89a9-27db7c1b4de6 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:01:02.809243 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 752/1496] 10.222.0.40 () {58 vars in 1153 bytes} [Fri Aug 26 11:01:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 122 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 11:01:02.817509 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514cf887-8a74-4f5b-960c-916f09a4b36d None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:01:02.827427 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514cf887-8a74-4f5b-960c-916f09a4b36d None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:01:02.892691 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 11:01:02.927286 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17e5706-125d-41dc-b26a-327ad685df70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:01:02.927890 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17e5706-125d-41dc-b26a-327ad685df70 admin admin] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:01:02.928164 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17e5706-125d-41dc-b26a-327ad685df70 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:01:02.928434 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17e5706-125d-41dc-b26a-327ad685df70 admin admin] PATH_INFO: `/v3/projects` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:01:02.929055 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a17e5706-125d-41dc-b26a-327ad685df70 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:01:02.930979 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: WARNING py.warnings [None req-a17e5706-125d-41dc-b26a-327ad685df70 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 26 11:01:02.930979 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: warnings.warn(msg) Aug 26 11:01:02.930979 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: Aug 26 11:01:02.932181 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a17e5706-125d-41dc-b26a-327ad685df70 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:01:02.987809 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 745/1497] 10.222.0.40 () {60 vars in 1153 bytes} [Fri Aug 26 11:01:02 2022] POST /identity/v3/projects => generated 377 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 11:01:02.992235 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020f395a-18a5-4343-89a9-27db7c1b4de6 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:01:02.994374 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-020f395a-18a5-4343-89a9-27db7c1b4de6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:01:03.027827 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020f395a-18a5-4343-89a9-27db7c1b4de6 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:01:03.120679 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0c31dc-9868-4bdf-af12-b2789e5023ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:01:03.121207 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0c31dc-9868-4bdf-af12-b2789e5023ad admin admin] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:01:03.121481 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0c31dc-9868-4bdf-af12-b2789e5023ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:01:03.121748 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0c31dc-9868-4bdf-af12-b2789e5023ad admin admin] PATH_INFO: `/v3/users` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:01:03.122414 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c0c31dc-9868-4bdf-af12-b2789e5023ad admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:01:03.123409 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: WARNING py.warnings [None req-5c0c31dc-9868-4bdf-af12-b2789e5023ad admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 26 11:01:03.123409 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: warnings.warn(msg) Aug 26 11:01:03.123409 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: Aug 26 11:01:03.124614 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c0c31dc-9868-4bdf-af12-b2789e5023ad admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:01:03.166584 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 753/1498] 10.222.0.40 () {60 vars in 1144 bytes} [Fri Aug 26 11:01:02 2022] POST /identity/v3/users => generated 289 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 11:01:03.170837 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17e5706-125d-41dc-b26a-327ad685df70 admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:01:03.172337 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-a17e5706-125d-41dc-b26a-327ad685df70 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:01:03.233453 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17e5706-125d-41dc-b26a-327ad685df70 admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:01:03.319115 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 11:01:03.331593 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ca8d37-fb7d-430f-8073-816b20bf3d92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:01:03.332200 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ca8d37-fb7d-430f-8073-816b20bf3d92 admin admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:01:03.332473 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ca8d37-fb7d-430f-8073-816b20bf3d92 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:01:03.332736 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ca8d37-fb7d-430f-8073-816b20bf3d92 admin admin] PATH_INFO: `/v3/roles` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:01:03.333367 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ca8d37-fb7d-430f-8073-816b20bf3d92 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:01:03.334397 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: WARNING py.warnings [None req-55ca8d37-fb7d-430f-8073-816b20bf3d92 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 26 11:01:03.334397 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: warnings.warn(msg) Aug 26 11:01:03.334397 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: Aug 26 11:01:03.335267 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ca8d37-fb7d-430f-8073-816b20bf3d92 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:01:03.347893 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 746/1499] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 26 11:01:03 2022] GET /identity/v3/roles => generated 1420 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 11:01:03.352120 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0c31dc-9868-4bdf-af12-b2789e5023ad admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:01:03.354176 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-5c0c31dc-9868-4bdf-af12-b2789e5023ad admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:01:03.365798 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0c31dc-9868-4bdf-af12-b2789e5023ad admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:01:03.391236 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 11:01:03.461527 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55edae9-ad4e-4ed8-ab6a-8b493933aff5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:01:03.462722 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55edae9-ad4e-4ed8-ab6a-8b493933aff5 admin admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:01:03.462998 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55edae9-ad4e-4ed8-ab6a-8b493933aff5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:01:03.463265 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55edae9-ad4e-4ed8-ab6a-8b493933aff5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:01:03.463904 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b55edae9-ad4e-4ed8-ab6a-8b493933aff5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:01:03.464903 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: WARNING py.warnings [None req-b55edae9-ad4e-4ed8-ab6a-8b493933aff5 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 26 11:01:03.464903 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: warnings.warn(msg) Aug 26 11:01:03.464903 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: Aug 26 11:01:03.465783 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b55edae9-ad4e-4ed8-ab6a-8b493933aff5 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:01:03.477814 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 754/1500] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 26 11:01:03 2022] GET /identity/v3/roles => generated 1420 bytes in 126 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 11:01:03.482041 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ca8d37-fb7d-430f-8073-816b20bf3d92 admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:01:03.483641 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-55ca8d37-fb7d-430f-8073-816b20bf3d92 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:01:03.517091 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ca8d37-fb7d-430f-8073-816b20bf3d92 admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:01:03.611843 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb82314-a666-441e-8cf0-8b8d1e7d3250 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:01:03.612496 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb82314-a666-441e-8cf0-8b8d1e7d3250 admin admin] REQUEST_METHOD: `PUT` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:01:03.612772 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb82314-a666-441e-8cf0-8b8d1e7d3250 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:01:03.613041 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb82314-a666-441e-8cf0-8b8d1e7d3250 admin admin] PATH_INFO: `/v3/projects/613653e83c69406dbd203a82d128ac9f/users/384b4bcfadea42cfad8e6cde3a847206/roles/23e174ce743b46eabe618c212691da55` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:01:03.636481 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb82314-a666-441e-8cf0-8b8d1e7d3250 admin admin] RBAC: Authorizing `identity:create_grant(project_id=613653e83c69406dbd203a82d128ac9f, user_id=384b4bcfadea42cfad8e6cde3a847206, role_id=23e174ce743b46eabe618c212691da55)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:01:03.637525 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: WARNING py.warnings [None req-1eb82314-a666-441e-8cf0-8b8d1e7d3250 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 26 11:01:03.637525 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: warnings.warn(msg) Aug 26 11:01:03.637525 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: Aug 26 11:01:03.639476 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb82314-a666-441e-8cf0-8b8d1e7d3250 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:01:03.664908 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 747/1501] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 26 11:01:03 2022] PUT /identity/v3/projects/613653e83c69406dbd203a82d128ac9f/users/384b4bcfadea42cfad8e6cde3a847206/roles/23e174ce743b46eabe618c212691da55 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 26 11:01:03.677451 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed510fb-e05c-4d65-a6e4-04e272bbff71 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:01:03.677733 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed510fb-e05c-4d65-a6e4-04e272bbff71 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:01:03.678029 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed510fb-e05c-4d65-a6e4-04e272bbff71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:01:03.733562 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-fed510fb-e05c-4d65-a6e4-04e272bbff71 None None] MFA Rules not processed for user `384b4bcfadea42cfad8e6cde3a847206`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 11:01:03.734823 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-fed510fb-e05c-4d65-a6e4-04e272bbff71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:01:03.810875 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 755/1502] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 11:01:03 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 11:01:03.824190 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fd00ca-7d62-475b-8a57-b27af84ee869 None None] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:01:03.824470 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fd00ca-7d62-475b-8a57-b27af84ee869 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:01:03.824740 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fd00ca-7d62-475b-8a57-b27af84ee869 None None] PATH_INFO: `/` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:01:03.826141 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 748/1503] 10.222.0.40 () {54 vars in 964 bytes} [Fri Aug 26 11:01:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 11:01:03.830958 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7679af70-c933-4272-82b9-7e356cd1e888 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:01:03.831245 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7679af70-c933-4272-82b9-7e356cd1e888 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:01:03.831523 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7679af70-c933-4272-82b9-7e356cd1e888 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:01:03.863478 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 11:01:03.907995 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-7679af70-c933-4272-82b9-7e356cd1e888 None None] MFA Rules not processed for user `c277338a7f814cc6b01c5419775031bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 11:01:03.909187 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-7679af70-c933-4272-82b9-7e356cd1e888 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:01:03.984163 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 756/1504] 10.222.0.40 () {58 vars in 1061 bytes} [Fri Aug 26 11:01:03 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 11:01:03.992242 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01fd00ca-7d62-475b-8a57-b27af84ee869 None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:01:03.994006 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-01fd00ca-7d62-475b-8a57-b27af84ee869 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:01:04.045871 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 11:01:04.100599 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01fd00ca-7d62-475b-8a57-b27af84ee869 None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:01:04.174047 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0aa799-33c3-4eea-af3b-ce063089a99f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c277338a7f814cc6b01c5419775031bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e30ccf0c1204727b786876c141dbcd2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:01:04.174651 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0aa799-33c3-4eea-af3b-ce063089a99f service cinder] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:01:04.174925 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0aa799-33c3-4eea-af3b-ce063089a99f service cinder] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:01:04.175192 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0aa799-33c3-4eea-af3b-ce063089a99f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:01:04.176791 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-ea0aa799-33c3-4eea-af3b-ce063089a99f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:01:04.275629 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea0aa799-33c3-4eea-af3b-ce063089a99f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:01:04.276985 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea0aa799-33c3-4eea-af3b-ce063089a99f service cinder] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:01:04.280495 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 11:01:04.295134 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 749/1505] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 11:01:03 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 11:01:04.340995 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3d2a81-d48d-46af-a7c5-f209cc0885f6 None None] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:01:04.341275 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3d2a81-d48d-46af-a7c5-f209cc0885f6 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:01:04.341544 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3d2a81-d48d-46af-a7c5-f209cc0885f6 None None] PATH_INFO: `/` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:01:04.342807 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 757/1506] 10.222.0.40 () {52 vars in 830 bytes} [Fri Aug 26 11:01:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 11:01:04.346324 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0aa799-33c3-4eea-af3b-ce063089a99f service cinder] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:01:04.347563 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-ea0aa799-33c3-4eea-af3b-ce063089a99f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:01:04.446982 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0aa799-33c3-4eea-af3b-ce063089a99f service cinder] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:01:04.544947 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee95dbe-db79-4d19-bd42-ed7aa51572b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:01:04.545548 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee95dbe-db79-4d19-bd42-ed7aa51572b8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:01:04.545775 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee95dbe-db79-4d19-bd42-ed7aa51572b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:01:04.546093 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee95dbe-db79-4d19-bd42-ed7aa51572b8 admin admin] PATH_INFO: `/v3/users/384b4bcfadea42cfad8e6cde3a847206` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:01:04.558317 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee95dbe-db79-4d19-bd42-ed7aa51572b8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=384b4bcfadea42cfad8e6cde3a847206)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:01:04.559364 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: WARNING py.warnings [None req-6ee95dbe-db79-4d19-bd42-ed7aa51572b8 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 26 11:01:04.559364 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: warnings.warn(msg) Aug 26 11:01:04.559364 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: Aug 26 11:01:04.560578 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee95dbe-db79-4d19-bd42-ed7aa51572b8 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:01:04.570272 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 11:01:04.627973 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-6ee95dbe-db79-4d19-bd42-ed7aa51572b8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '384b4bcfadea42cfad8e6cde3a847206'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:01:04.643342 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-6ee95dbe-db79-4d19-bd42-ed7aa51572b8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '384b4bcfadea42cfad8e6cde3a847206'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:01:04.643887 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-6ee95dbe-db79-4d19-bd42-ed7aa51572b8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '384b4bcfadea42cfad8e6cde3a847206'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:01:04.667352 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-6ee95dbe-db79-4d19-bd42-ed7aa51572b8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '384b4bcfadea42cfad8e6cde3a847206'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:01:04.667813 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-6ee95dbe-db79-4d19-bd42-ed7aa51572b8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '384b4bcfadea42cfad8e6cde3a847206'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:01:04.682686 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 750/1507] 10.222.0.40 () {58 vars in 1224 bytes} [Fri Aug 26 11:01:04 2022] DELETE /identity/v3/users/384b4bcfadea42cfad8e6cde3a847206 => generated 0 bytes in 337 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 11:01:04.687076 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f21f590-4152-42c5-b57c-1f8d7dd37fc9 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:01:04.687575 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f21f590-4152-42c5-b57c-1f8d7dd37fc9 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:01:04.687716 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f21f590-4152-42c5-b57c-1f8d7dd37fc9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:01:04.745778 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-7f21f590-4152-42c5-b57c-1f8d7dd37fc9 None None] MFA Rules not processed for user `ca155a48e2024c32af3fd2e19e5fac11`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 11:01:04.747059 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-7f21f590-4152-42c5-b57c-1f8d7dd37fc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:01:04.820197 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 11:01:04.832887 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 758/1508] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 11:01:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 11:01:04.843035 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee95dbe-db79-4d19-bd42-ed7aa51572b8 admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:01:04.844259 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-6ee95dbe-db79-4d19-bd42-ed7aa51572b8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:01:04.946197 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee95dbe-db79-4d19-bd42-ed7aa51572b8 admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:01:05.000279 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 11:01:05.020728 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b069859-8c2f-4037-97f8-fbc7b185aa1f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91da2acd2534423091537e1ba068d2b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e30ccf0c1204727b786876c141dbcd2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:01:05.021342 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b069859-8c2f-4037-97f8-fbc7b185aa1f service neutron] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:01:05.021646 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b069859-8c2f-4037-97f8-fbc7b185aa1f service neutron] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:01:05.021916 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b069859-8c2f-4037-97f8-fbc7b185aa1f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:01:05.023597 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-2b069859-8c2f-4037-97f8-fbc7b185aa1f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:01:05.131946 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b069859-8c2f-4037-97f8-fbc7b185aa1f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:01:05.133386 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b069859-8c2f-4037-97f8-fbc7b185aa1f service neutron] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:01:05.151962 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 751/1509] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 11:01:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 11:01:05.513087 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f21f590-4152-42c5-b57c-1f8d7dd37fc9 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:01:05.514417 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-7f21f590-4152-42c5-b57c-1f8d7dd37fc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:01:05.530543 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f21f590-4152-42c5-b57c-1f8d7dd37fc9 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:01:05.628739 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94cd2fb-7991-4073-851e-9b6978264e31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:01:05.629370 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94cd2fb-7991-4073-851e-9b6978264e31 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:01:05.629644 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94cd2fb-7991-4073-851e-9b6978264e31 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:01:05.629910 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94cd2fb-7991-4073-851e-9b6978264e31 admin admin] PATH_INFO: `/v3/projects/613653e83c69406dbd203a82d128ac9f` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:01:05.639757 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94cd2fb-7991-4073-851e-9b6978264e31 admin admin] RBAC: Authorizing `identity:delete_project(project_id=613653e83c69406dbd203a82d128ac9f)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:01:05.640850 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: WARNING py.warnings [None req-d94cd2fb-7991-4073-851e-9b6978264e31 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 26 11:01:05.640850 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: warnings.warn(msg) Aug 26 11:01:05.640850 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: Aug 26 11:01:05.642004 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94cd2fb-7991-4073-851e-9b6978264e31 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:01:05.698686 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-d94cd2fb-7991-4073-851e-9b6978264e31 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '613653e83c69406dbd203a82d128ac9f', 'request_id': 'req-d94cd2fb-7991-4073-851e-9b6978264e31'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:01:05.711625 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-d94cd2fb-7991-4073-851e-9b6978264e31 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '613653e83c69406dbd203a82d128ac9f', 'request_id': 'req-d94cd2fb-7991-4073-851e-9b6978264e31'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:01:05.717787 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-d94cd2fb-7991-4073-851e-9b6978264e31 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '613653e83c69406dbd203a82d128ac9f', 'request_id': 'req-d94cd2fb-7991-4073-851e-9b6978264e31'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:01:05.744890 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-d94cd2fb-7991-4073-851e-9b6978264e31 admin admin] The token cache is being invalidate because project 613653e83c69406dbd203a82d128ac9f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81424) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 11:01:05.745166 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-d94cd2fb-7991-4073-851e-9b6978264e31 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:01:05.745680 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-d94cd2fb-7991-4073-851e-9b6978264e31 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:01:05.747044 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 759/1510] 10.222.0.40 () {58 vars in 1233 bytes} [Fri Aug 26 11:01:05 2022] DELETE /identity/v3/projects/613653e83c69406dbd203a82d128ac9f => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 11:01:15.149422 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=62 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 11:01:15.742350 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 11:02:36.433397 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8c8829-ddf5-41b3-94a4-185e0d0e2691 None None] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:02:36.434023 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8c8829-ddf5-41b3-94a4-185e0d0e2691 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:02:36.434023 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8c8829-ddf5-41b3-94a4-185e0d0e2691 None None] PATH_INFO: `/` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:02:36.435312 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 752/1511] 127.0.0.1 () {54 vars in 901 bytes} [Fri Aug 26 11:02:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 11:02:36.440289 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b16615-657c-4c69-b5bc-f4963eb489db None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:02:36.440712 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b16615-657c-4c69-b5bc-f4963eb489db None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:02:36.440822 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b16615-657c-4c69-b5bc-f4963eb489db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:02:36.521801 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-e9b16615-657c-4c69-b5bc-f4963eb489db None None] MFA Rules not processed for user `ca155a48e2024c32af3fd2e19e5fac11`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 11:02:36.523703 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-e9b16615-657c-4c69-b5bc-f4963eb489db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:02:36.627978 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 760/1512] 127.0.0.1 () {58 vars in 998 bytes} [Fri Aug 26 11:02:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 11:02:36.783245 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8c8829-ddf5-41b3-94a4-185e0d0e2691 None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:02:36.785600 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-1f8c8829-ddf5-41b3-94a4-185e0d0e2691 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:02:36.911349 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8c8829-ddf5-41b3-94a4-185e0d0e2691 None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:02:36.918059 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d38a435-0da7-4591-8170-3c3ab790d78f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f4b4a662db049ecacc865ecff10bdb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e30ccf0c1204727b786876c141dbcd2', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:02:36.918668 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d38a435-0da7-4591-8170-3c3ab790d78f service nova] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:02:36.918942 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d38a435-0da7-4591-8170-3c3ab790d78f service nova] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:02:36.919209 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d38a435-0da7-4591-8170-3c3ab790d78f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:02:36.928741 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d38a435-0da7-4591-8170-3c3ab790d78f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:02:36.930379 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d38a435-0da7-4591-8170-3c3ab790d78f service nova] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:02:36.939963 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 753/1513] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 11:02:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 11:02:37.769800 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82708d9d-e5f6-43d3-b4a9-3055f2132f69 None None] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:02:37.770393 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82708d9d-e5f6-43d3-b4a9-3055f2132f69 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:02:37.770393 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82708d9d-e5f6-43d3-b4a9-3055f2132f69 None None] PATH_INFO: `/` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:02:37.771762 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 761/1514] 127.0.0.1 () {54 vars in 901 bytes} [Fri Aug 26 11:02:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 11:02:37.776989 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9d2304-b2a7-4bbf-8f0c-b8a360d8060d None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:02:37.777409 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9d2304-b2a7-4bbf-8f0c-b8a360d8060d None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:02:37.777541 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9d2304-b2a7-4bbf-8f0c-b8a360d8060d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:02:37.798668 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-ec9d2304-b2a7-4bbf-8f0c-b8a360d8060d None None] MFA Rules not processed for user `ca155a48e2024c32af3fd2e19e5fac11`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 11:02:37.799865 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-ec9d2304-b2a7-4bbf-8f0c-b8a360d8060d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:02:37.807909 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 754/1515] 127.0.0.1 () {58 vars in 998 bytes} [Fri Aug 26 11:02:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 11:02:37.961242 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82708d9d-e5f6-43d3-b4a9-3055f2132f69 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:02:37.975598 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82708d9d-e5f6-43d3-b4a9-3055f2132f69 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:02:37.982086 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92406a07-9ac3-4edd-a6dc-8c0356e7b70a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f4b4a662db049ecacc865ecff10bdb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e30ccf0c1204727b786876c141dbcd2', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:02:37.982931 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92406a07-9ac3-4edd-a6dc-8c0356e7b70a service nova] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:02:37.983425 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92406a07-9ac3-4edd-a6dc-8c0356e7b70a service nova] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:02:37.983902 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92406a07-9ac3-4edd-a6dc-8c0356e7b70a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:02:37.993340 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92406a07-9ac3-4edd-a6dc-8c0356e7b70a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:02:37.994982 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92406a07-9ac3-4edd-a6dc-8c0356e7b70a service nova] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:02:38.005086 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 762/1516] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 11:02:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 11:02:39.527149 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269adbd2-0557-49ad-ac4d-539d48b81eef None None] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:02:39.527713 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269adbd2-0557-49ad-ac4d-539d48b81eef None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:02:39.527713 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269adbd2-0557-49ad-ac4d-539d48b81eef None None] PATH_INFO: `/v3` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:02:39.529006 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 755/1517] 10.222.0.26 () {54 vars in 925 bytes} [Fri Aug 26 11:02:39 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 11:02:39.535745 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6abec2-0ff0-47f7-b2e7-a79da4b285f6 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:02:39.536141 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6abec2-0ff0-47f7-b2e7-a79da4b285f6 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:02:39.536288 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6abec2-0ff0-47f7-b2e7-a79da4b285f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:02:39.594220 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-ec6abec2-0ff0-47f7-b2e7-a79da4b285f6 None None] MFA Rules not processed for user `f29c5dfed31f42a9b5cbc0cd72e9fdc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 11:02:39.595429 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-ec6abec2-0ff0-47f7-b2e7-a79da4b285f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:02:39.616230 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 11:02:39.671612 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 763/1518] 10.222.0.26 () {58 vars in 1015 bytes} [Fri Aug 26 11:02:39 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 11:02:43.797318 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f70489e-a66c-4cfe-adea-97b91783f2d3 None None] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:02:43.797911 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f70489e-a66c-4cfe-adea-97b91783f2d3 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:02:43.797911 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f70489e-a66c-4cfe-adea-97b91783f2d3 None None] PATH_INFO: `/` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:02:43.799289 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 756/1519] 10.222.0.40 () {54 vars in 961 bytes} [Fri Aug 26 11:02:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 11:02:43.804932 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d38c0dc-72e2-4bd2-8d0b-dbec619682b2 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:02:43.805371 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d38c0dc-72e2-4bd2-8d0b-dbec619682b2 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:02:43.805469 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d38c0dc-72e2-4bd2-8d0b-dbec619682b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:02:43.825446 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-2d38c0dc-72e2-4bd2-8d0b-dbec619682b2 None None] MFA Rules not processed for user `f29c5dfed31f42a9b5cbc0cd72e9fdc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 11:02:43.826673 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-2d38c0dc-72e2-4bd2-8d0b-dbec619682b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:02:43.835157 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 764/1520] 10.222.0.40 () {58 vars in 1058 bytes} [Fri Aug 26 11:02:43 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 11:02:43.842714 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f70489e-a66c-4cfe-adea-97b91783f2d3 None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:02:43.857024 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f70489e-a66c-4cfe-adea-97b91783f2d3 None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:02:43.862120 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ba6a02-16a9-474a-b62e-f11720306044 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f29c5dfed31f42a9b5cbc0cd72e9fdc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e30ccf0c1204727b786876c141dbcd2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:02:43.862714 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ba6a02-16a9-474a-b62e-f11720306044 service placement] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:02:43.862992 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ba6a02-16a9-474a-b62e-f11720306044 service placement] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:02:43.863264 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ba6a02-16a9-474a-b62e-f11720306044 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:02:43.872719 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ba6a02-16a9-474a-b62e-f11720306044 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:02:43.874324 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ba6a02-16a9-474a-b62e-f11720306044 service placement] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:02:43.884174 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 757/1521] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 11:02:43 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 11:02:49.247255 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea02c0cf-f8d6-4197-a046-15f8c7a83e4a None None] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:02:49.247790 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea02c0cf-f8d6-4197-a046-15f8c7a83e4a None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:02:49.247790 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea02c0cf-f8d6-4197-a046-15f8c7a83e4a None None] PATH_INFO: `/` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:02:49.249034 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 765/1522] 127.0.0.1 () {54 vars in 901 bytes} [Fri Aug 26 11:02:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 11:02:49.253734 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86f2ede-f67e-4b1d-9136-edad4b8d344b None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:02:49.254188 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86f2ede-f67e-4b1d-9136-edad4b8d344b None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:02:49.254339 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86f2ede-f67e-4b1d-9136-edad4b8d344b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:02:49.273719 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-c86f2ede-f67e-4b1d-9136-edad4b8d344b None None] MFA Rules not processed for user `ca155a48e2024c32af3fd2e19e5fac11`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 11:02:49.274982 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-c86f2ede-f67e-4b1d-9136-edad4b8d344b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:02:49.282952 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 758/1523] 127.0.0.1 () {58 vars in 998 bytes} [Fri Aug 26 11:02:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 11:02:49.439532 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea02c0cf-f8d6-4197-a046-15f8c7a83e4a None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:02:49.455001 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea02c0cf-f8d6-4197-a046-15f8c7a83e4a None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:02:49.460982 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd9c6b7-5f2d-4a4e-8f65-90093af67ac4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f4b4a662db049ecacc865ecff10bdb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e30ccf0c1204727b786876c141dbcd2', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:02:49.461575 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd9c6b7-5f2d-4a4e-8f65-90093af67ac4 service nova] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:02:49.461851 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd9c6b7-5f2d-4a4e-8f65-90093af67ac4 service nova] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:02:49.462390 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd9c6b7-5f2d-4a4e-8f65-90093af67ac4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:02:49.470471 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd9c6b7-5f2d-4a4e-8f65-90093af67ac4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:02:49.472045 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd9c6b7-5f2d-4a4e-8f65-90093af67ac4 service nova] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:02:49.481378 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 766/1524] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 11:02:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 11:02:50.306831 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a6c60f-cfea-471f-939e-b59a906865b4 None None] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:02:50.307369 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a6c60f-cfea-471f-939e-b59a906865b4 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:02:50.307369 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a6c60f-cfea-471f-939e-b59a906865b4 None None] PATH_INFO: `/` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:02:50.308659 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 759/1525] 127.0.0.1 () {54 vars in 901 bytes} [Fri Aug 26 11:02:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 11:02:50.313593 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ea250b-0fcc-4f32-9fcb-959b2befbd2a None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:02:50.314067 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ea250b-0fcc-4f32-9fcb-959b2befbd2a None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:02:50.314174 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ea250b-0fcc-4f32-9fcb-959b2befbd2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:02:50.334146 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-e7ea250b-0fcc-4f32-9fcb-959b2befbd2a None None] MFA Rules not processed for user `ca155a48e2024c32af3fd2e19e5fac11`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 11:02:50.335351 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-e7ea250b-0fcc-4f32-9fcb-959b2befbd2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:02:50.345732 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 767/1526] 127.0.0.1 () {58 vars in 998 bytes} [Fri Aug 26 11:02:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 11:02:50.501700 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a6c60f-cfea-471f-939e-b59a906865b4 None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:02:50.516699 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a6c60f-cfea-471f-939e-b59a906865b4 None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:02:50.522153 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6d49af-7fc0-409f-8384-e387eb34799b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f4b4a662db049ecacc865ecff10bdb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e30ccf0c1204727b786876c141dbcd2', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:02:50.522740 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6d49af-7fc0-409f-8384-e387eb34799b service nova] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:02:50.523014 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6d49af-7fc0-409f-8384-e387eb34799b service nova] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:02:50.523282 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6d49af-7fc0-409f-8384-e387eb34799b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:02:50.531269 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa6d49af-7fc0-409f-8384-e387eb34799b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:02:50.532843 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa6d49af-7fc0-409f-8384-e387eb34799b service nova] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:02:50.542690 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 760/1527] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 11:02:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 11:02:54.311168 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b6ad3a-1e9d-49a9-a40c-9ee8de0d7c0a None None] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:02:54.311771 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b6ad3a-1e9d-49a9-a40c-9ee8de0d7c0a None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:02:54.311771 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b6ad3a-1e9d-49a9-a40c-9ee8de0d7c0a None None] PATH_INFO: `/` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:02:54.312984 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 768/1528] 127.0.0.1 () {54 vars in 917 bytes} [Fri Aug 26 11:02:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 11:02:54.318850 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead21752-a2da-45a0-b885-fb030cac5e6a None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:02:54.319327 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead21752-a2da-45a0-b885-fb030cac5e6a None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:02:54.319441 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead21752-a2da-45a0-b885-fb030cac5e6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:02:54.340373 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-ead21752-a2da-45a0-b885-fb030cac5e6a None None] MFA Rules not processed for user `ca155a48e2024c32af3fd2e19e5fac11`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 11:02:54.341626 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-ead21752-a2da-45a0-b885-fb030cac5e6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:02:54.349856 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 761/1529] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Aug 26 11:02:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 11:02:54.664581 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b6ad3a-1e9d-49a9-a40c-9ee8de0d7c0a None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:02:54.666774 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-b7b6ad3a-1e9d-49a9-a40c-9ee8de0d7c0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:02:54.744925 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b6ad3a-1e9d-49a9-a40c-9ee8de0d7c0a None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:02:54.748589 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd490a3-f525-4d25-a687-2acd2a6e193b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91da2acd2534423091537e1ba068d2b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e30ccf0c1204727b786876c141dbcd2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:02:54.749197 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd490a3-f525-4d25-a687-2acd2a6e193b service neutron] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:02:54.749473 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd490a3-f525-4d25-a687-2acd2a6e193b service neutron] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:02:54.749743 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd490a3-f525-4d25-a687-2acd2a6e193b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:02:54.759510 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd490a3-f525-4d25-a687-2acd2a6e193b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:02:54.760933 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd490a3-f525-4d25-a687-2acd2a6e193b service neutron] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:02:54.770341 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 769/1530] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 11:02:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 11:02:56.678842 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d618f49-72f6-4155-96e4-e0217a9d24cc None None] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:02:56.679385 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d618f49-72f6-4155-96e4-e0217a9d24cc None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:02:56.679385 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d618f49-72f6-4155-96e4-e0217a9d24cc None None] PATH_INFO: `/` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:02:56.681364 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 762/1531] 127.0.0.1 () {54 vars in 917 bytes} [Fri Aug 26 11:02:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 11:02:56.686615 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea72137b-f57e-40ad-b223-2b1e4c870dac None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:02:56.687021 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea72137b-f57e-40ad-b223-2b1e4c870dac None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:02:56.687169 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea72137b-f57e-40ad-b223-2b1e4c870dac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:02:56.706331 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-ea72137b-f57e-40ad-b223-2b1e4c870dac None None] MFA Rules not processed for user `ca155a48e2024c32af3fd2e19e5fac11`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 11:02:56.707536 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-ea72137b-f57e-40ad-b223-2b1e4c870dac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:02:56.715523 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 770/1532] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Aug 26 11:02:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 11:02:57.004592 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d618f49-72f6-4155-96e4-e0217a9d24cc None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:02:57.016189 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d618f49-72f6-4155-96e4-e0217a9d24cc None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:02:57.019743 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6c16c2-931a-4776-bbf6-06e71de325ef service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91da2acd2534423091537e1ba068d2b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e30ccf0c1204727b786876c141dbcd2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:02:57.020328 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6c16c2-931a-4776-bbf6-06e71de325ef service neutron] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:02:57.020604 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6c16c2-931a-4776-bbf6-06e71de325ef service neutron] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:02:57.020875 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6c16c2-931a-4776-bbf6-06e71de325ef service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:02:57.029692 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f6c16c2-931a-4776-bbf6-06e71de325ef service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:02:57.031147 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f6c16c2-931a-4776-bbf6-06e71de325ef service neutron] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:02:57.040504 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 763/1533] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 11:02:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 11:03:00.454265 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124419f8-34ba-4a6b-bde1-52ec88d552e0 None None] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:03:00.454842 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124419f8-34ba-4a6b-bde1-52ec88d552e0 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:03:00.454842 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124419f8-34ba-4a6b-bde1-52ec88d552e0 None None] PATH_INFO: `/` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:03:00.456128 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 771/1534] 127.0.0.1 () {54 vars in 917 bytes} [Fri Aug 26 11:03:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 11:03:00.461217 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f374e88-6ec1-4744-a57f-7fd2496cc746 None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:03:00.461749 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f374e88-6ec1-4744-a57f-7fd2496cc746 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:03:00.461900 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f374e88-6ec1-4744-a57f-7fd2496cc746 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:03:00.481681 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-8f374e88-6ec1-4744-a57f-7fd2496cc746 None None] MFA Rules not processed for user `ca155a48e2024c32af3fd2e19e5fac11`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 11:03:00.482924 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-8f374e88-6ec1-4744-a57f-7fd2496cc746 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:03:00.491329 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 764/1535] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Aug 26 11:03:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 11:03:01.050646 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124419f8-34ba-4a6b-bde1-52ec88d552e0 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:03:01.052320 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-124419f8-34ba-4a6b-bde1-52ec88d552e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:03:01.118800 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 11:03:01.132320 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124419f8-34ba-4a6b-bde1-52ec88d552e0 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:03:01.136339 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7033e1-0013-4dc9-a52f-9b7fda2610d3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c277338a7f814cc6b01c5419775031bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e30ccf0c1204727b786876c141dbcd2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:03:01.136942 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7033e1-0013-4dc9-a52f-9b7fda2610d3 service cinder] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:03:01.137220 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7033e1-0013-4dc9-a52f-9b7fda2610d3 service cinder] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:03:01.137492 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7033e1-0013-4dc9-a52f-9b7fda2610d3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:03:01.146403 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a7033e1-0013-4dc9-a52f-9b7fda2610d3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:03:01.147827 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a7033e1-0013-4dc9-a52f-9b7fda2610d3 service cinder] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:03:01.157556 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 772/1536] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 11:03:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 11:03:03.103187 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca00dfb8-06fd-4499-8525-a61db9219333 None None] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:03:03.103719 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca00dfb8-06fd-4499-8525-a61db9219333 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:03:03.103719 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca00dfb8-06fd-4499-8525-a61db9219333 None None] PATH_INFO: `/` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:03:03.105089 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 765/1537] 127.0.0.1 () {54 vars in 917 bytes} [Fri Aug 26 11:03:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 11:03:03.110311 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f64c35-f6ba-4599-9312-f854fda19012 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:03:03.110735 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f64c35-f6ba-4599-9312-f854fda19012 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:03:03.110839 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f64c35-f6ba-4599-9312-f854fda19012 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:03:03.130361 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-32f64c35-f6ba-4599-9312-f854fda19012 None None] MFA Rules not processed for user `ca155a48e2024c32af3fd2e19e5fac11`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 11:03:03.131482 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-32f64c35-f6ba-4599-9312-f854fda19012 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:03:03.139823 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 773/1538] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Aug 26 11:03:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 11:03:03.700099 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca00dfb8-06fd-4499-8525-a61db9219333 None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:03:03.701777 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-ca00dfb8-06fd-4499-8525-a61db9219333 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:03:03.715031 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca00dfb8-06fd-4499-8525-a61db9219333 None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:03:03.718591 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8842d344-a8b9-4503-a381-14341d4ccb4f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c277338a7f814cc6b01c5419775031bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e30ccf0c1204727b786876c141dbcd2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:03:03.719185 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8842d344-a8b9-4503-a381-14341d4ccb4f service cinder] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:03:03.719457 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8842d344-a8b9-4503-a381-14341d4ccb4f service cinder] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:03:03.719723 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8842d344-a8b9-4503-a381-14341d4ccb4f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:03:03.728503 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8842d344-a8b9-4503-a381-14341d4ccb4f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:03:03.729911 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8842d344-a8b9-4503-a381-14341d4ccb4f service cinder] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:03:03.739718 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 766/1539] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 11:03:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 26 11:03:13.125719 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=11 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 11:03:13.735031 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=89 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 11:04:28.865461 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16986d5e-5499-4d57-b99b-c6895a235259 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:04:28.916785 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16986d5e-5499-4d57-b99b-c6895a235259 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:04:28.916785 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16986d5e-5499-4d57-b99b-c6895a235259 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:04:28.916785 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-16986d5e-5499-4d57-b99b-c6895a235259 None None] MFA Rules not processed for user `ca155a48e2024c32af3fd2e19e5fac11`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 11:04:28.916785 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-16986d5e-5499-4d57-b99b-c6895a235259 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:04:28.931078 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 774/1540] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 11:04:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 66 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 11:04:28.940140 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abc2c01-bfc8-4c0f-af73-7b877872f703 None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:04:28.944539 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abc2c01-bfc8-4c0f-af73-7b877872f703 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:04:28.944539 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abc2c01-bfc8-4c0f-af73-7b877872f703 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:04:28.968083 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-3abc2c01-bfc8-4c0f-af73-7b877872f703 None None] MFA Rules not processed for user `ca155a48e2024c32af3fd2e19e5fac11`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 11:04:28.970059 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-3abc2c01-bfc8-4c0f-af73-7b877872f703 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:04:28.987179 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 767/1541] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 11:04:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 11:04:28.992843 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16986d5e-5499-4d57-b99b-c6895a235259 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:04:29.006342 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16986d5e-5499-4d57-b99b-c6895a235259 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:04:29.014502 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebbe2616-9fc2-4061-a1ff-7a278481c3af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:04:29.015147 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbe2616-9fc2-4061-a1ff-7a278481c3af admin admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:04:29.015427 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbe2616-9fc2-4061-a1ff-7a278481c3af admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:04:29.015701 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbe2616-9fc2-4061-a1ff-7a278481c3af admin admin] PATH_INFO: `/v3/domains` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:04:29.016372 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebbe2616-9fc2-4061-a1ff-7a278481c3af admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 11:04:29.016675 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebbe2616-9fc2-4061-a1ff-7a278481c3af admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:04:29.018379 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebbe2616-9fc2-4061-a1ff-7a278481c3af admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:04:29.034722 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 775/1542] 10.222.0.40 () {58 vars in 1153 bytes} [Fri Aug 26 11:04:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 11:04:29.043927 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3abc2c01-bfc8-4c0f-af73-7b877872f703 None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:04:29.059102 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3abc2c01-bfc8-4c0f-af73-7b877872f703 None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:04:29.066621 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3587aced-aa40-4a88-82f0-26bfcd6b6463 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:04:29.067248 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3587aced-aa40-4a88-82f0-26bfcd6b6463 admin admin] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:04:29.067567 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3587aced-aa40-4a88-82f0-26bfcd6b6463 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:04:29.067840 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3587aced-aa40-4a88-82f0-26bfcd6b6463 admin admin] PATH_INFO: `/v3/projects` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:04:29.068530 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3587aced-aa40-4a88-82f0-26bfcd6b6463 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:04:29.070849 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3587aced-aa40-4a88-82f0-26bfcd6b6463 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:04:29.128042 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 768/1543] 10.222.0.40 () {60 vars in 1153 bytes} [Fri Aug 26 11:04:29 2022] POST /identity/v3/projects => generated 371 bytes in 84 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 26 11:04:29.132154 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebbe2616-9fc2-4061-a1ff-7a278481c3af admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:04:29.143481 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebbe2616-9fc2-4061-a1ff-7a278481c3af admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:04:29.197904 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdba729-f37d-44e6-8995-7cf96e18ffef None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:04:29.198387 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdba729-f37d-44e6-8995-7cf96e18ffef None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:04:29.198656 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdba729-f37d-44e6-8995-7cf96e18ffef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:04:29.238047 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-3cdba729-f37d-44e6-8995-7cf96e18ffef None None] MFA Rules not processed for user `ca155a48e2024c32af3fd2e19e5fac11`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 11:04:29.239521 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-3cdba729-f37d-44e6-8995-7cf96e18ffef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:04:29.271880 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4fbd1d9-2714-43bf-82d7-1a94bc417e77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:04:29.274060 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fbd1d9-2714-43bf-82d7-1a94bc417e77 admin admin] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:04:29.274585 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fbd1d9-2714-43bf-82d7-1a94bc417e77 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:04:29.275088 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fbd1d9-2714-43bf-82d7-1a94bc417e77 admin admin] PATH_INFO: `/v3/users` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:04:29.275956 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4fbd1d9-2714-43bf-82d7-1a94bc417e77 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:04:29.282362 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4fbd1d9-2714-43bf-82d7-1a94bc417e77 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:04:29.322754 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 776/1544] 10.222.0.40 () {60 vars in 1144 bytes} [Fri Aug 26 11:04:29 2022] POST /identity/v3/users => generated 286 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 11:04:29.327128 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4fbd1d9-2714-43bf-82d7-1a94bc417e77 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:04:29.349016 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4fbd1d9-2714-43bf-82d7-1a94bc417e77 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:04:29.360838 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 769/1545] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 11:04:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (3 switches on core 0) Aug 26 11:04:29.363153 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6aebe3-9217-4328-92ff-74a8441b8f6b None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:04:29.364094 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d146ffc3-c5bc-4c7a-bca5-86c1a32cace4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:04:29.364960 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d146ffc3-c5bc-4c7a-bca5-86c1a32cace4 admin admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:04:29.365471 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d146ffc3-c5bc-4c7a-bca5-86c1a32cace4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:04:29.366014 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d146ffc3-c5bc-4c7a-bca5-86c1a32cace4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:04:29.375552 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d146ffc3-c5bc-4c7a-bca5-86c1a32cace4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:04:29.375902 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6aebe3-9217-4328-92ff-74a8441b8f6b None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:04:29.376257 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6aebe3-9217-4328-92ff-74a8441b8f6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:04:29.378286 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d146ffc3-c5bc-4c7a-bca5-86c1a32cace4 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:04:29.404043 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 777/1546] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 26 11:04:29 2022] GET /identity/v3/roles => generated 1420 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 26 11:04:29.405403 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7756269b-2151-400a-bf00-d7be41bc4dbc None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:04:29.405690 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7756269b-2151-400a-bf00-d7be41bc4dbc None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:04:29.405966 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7756269b-2151-400a-bf00-d7be41bc4dbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:04:29.410011 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-7d6aebe3-9217-4328-92ff-74a8441b8f6b None None] MFA Rules not processed for user `ca155a48e2024c32af3fd2e19e5fac11`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 11:04:29.411272 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-7d6aebe3-9217-4328-92ff-74a8441b8f6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:04:29.423613 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 770/1547] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 11:04:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 61 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 11:04:29.425791 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad41ab2f-4e93-48e3-9b92-823fe0056548 None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:04:29.426100 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad41ab2f-4e93-48e3-9b92-823fe0056548 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:04:29.426370 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad41ab2f-4e93-48e3-9b92-823fe0056548 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:04:29.452451 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-7756269b-2151-400a-bf00-d7be41bc4dbc None None] MFA Rules not processed for user `ca155a48e2024c32af3fd2e19e5fac11`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 11:04:29.453751 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-7756269b-2151-400a-bf00-d7be41bc4dbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:04:29.465380 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-ad41ab2f-4e93-48e3-9b92-823fe0056548 None None] MFA Rules not processed for user `ca155a48e2024c32af3fd2e19e5fac11`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 11:04:29.466402 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 778/1548] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 11:04:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 62 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 11:04:29.466671 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-ad41ab2f-4e93-48e3-9b92-823fe0056548 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:04:29.467638 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88625eb8-9561-48f1-b92e-8bc7c4ba932d None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:04:29.467919 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88625eb8-9561-48f1-b92e-8bc7c4ba932d None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:04:29.468191 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88625eb8-9561-48f1-b92e-8bc7c4ba932d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:04:29.475772 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 771/1549] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 11:04:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 11:04:29.477267 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad41ab2f-4e93-48e3-9b92-823fe0056548 None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:04:29.487698 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad41ab2f-4e93-48e3-9b92-823fe0056548 None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:04:29.494331 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a949065e-59ab-4c0d-bdab-5416c9b3a5e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:04:29.494984 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a949065e-59ab-4c0d-bdab-5416c9b3a5e6 admin admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:04:29.495264 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a949065e-59ab-4c0d-bdab-5416c9b3a5e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:04:29.495530 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a949065e-59ab-4c0d-bdab-5416c9b3a5e6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:04:29.496742 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a949065e-59ab-4c0d-bdab-5416c9b3a5e6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:04:29.498416 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a949065e-59ab-4c0d-bdab-5416c9b3a5e6 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:04:29.505858 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-88625eb8-9561-48f1-b92e-8bc7c4ba932d None None] MFA Rules not processed for user `ca155a48e2024c32af3fd2e19e5fac11`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 11:04:29.505858 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-88625eb8-9561-48f1-b92e-8bc7c4ba932d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:04:29.512034 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 772/1550] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 26 11:04:29 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 11:04:29.512184 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 779/1551] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 11:04:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 11:04:29.513626 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80eaf37-1a0d-4d4b-b4b5-0d75218eabe4 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:04:29.513909 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80eaf37-1a0d-4d4b-b4b5-0d75218eabe4 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:04:29.514210 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80eaf37-1a0d-4d4b-b4b5-0d75218eabe4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:04:29.516731 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67adef7d-6269-4684-8d49-e4c74f857bf4 None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:04:29.517012 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67adef7d-6269-4684-8d49-e4c74f857bf4 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:04:29.517285 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67adef7d-6269-4684-8d49-e4c74f857bf4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:04:29.537511 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-67adef7d-6269-4684-8d49-e4c74f857bf4 None None] MFA Rules not processed for user `ca155a48e2024c32af3fd2e19e5fac11`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 11:04:29.538780 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-67adef7d-6269-4684-8d49-e4c74f857bf4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:04:29.539482 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-b80eaf37-1a0d-4d4b-b4b5-0d75218eabe4 None None] MFA Rules not processed for user `ca155a48e2024c32af3fd2e19e5fac11`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 11:04:29.540708 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-b80eaf37-1a0d-4d4b-b4b5-0d75218eabe4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:04:29.550549 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 773/1552] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 11:04:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 11:04:29.550549 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67adef7d-6269-4684-8d49-e4c74f857bf4 None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:04:29.552623 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 780/1553] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 11:04:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 11:04:29.553877 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12575963-d68a-4aa1-9b60-11d42c7f6bc9 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:04:29.554745 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12575963-d68a-4aa1-9b60-11d42c7f6bc9 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:04:29.554745 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12575963-d68a-4aa1-9b60-11d42c7f6bc9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:04:29.558112 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67adef7d-6269-4684-8d49-e4c74f857bf4 None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:04:29.562821 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849fc094-46ad-4eb7-8201-f4814eb53a7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:04:29.563412 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849fc094-46ad-4eb7-8201-f4814eb53a7d admin admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:04:29.563682 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849fc094-46ad-4eb7-8201-f4814eb53a7d admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:04:29.563953 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849fc094-46ad-4eb7-8201-f4814eb53a7d admin admin] PATH_INFO: `/v3/domains` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:04:29.564589 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-849fc094-46ad-4eb7-8201-f4814eb53a7d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 11:04:29.564883 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-849fc094-46ad-4eb7-8201-f4814eb53a7d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:04:29.565893 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: WARNING py.warnings [None req-849fc094-46ad-4eb7-8201-f4814eb53a7d admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 26 11:04:29.565893 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: warnings.warn(msg) Aug 26 11:04:29.565893 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: Aug 26 11:04:29.566865 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-849fc094-46ad-4eb7-8201-f4814eb53a7d admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:04:29.582024 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 774/1554] 10.222.0.40 () {58 vars in 1153 bytes} [Fri Aug 26 11:04:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 11:04:29.582024 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901c24f8-5dc9-4930-bdac-bc4335c1eaac None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:04:29.582024 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901c24f8-5dc9-4930-bdac-bc4335c1eaac None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:04:29.582024 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901c24f8-5dc9-4930-bdac-bc4335c1eaac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:04:29.586330 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-12575963-d68a-4aa1-9b60-11d42c7f6bc9 None None] MFA Rules not processed for user `ca155a48e2024c32af3fd2e19e5fac11`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 11:04:29.586330 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-12575963-d68a-4aa1-9b60-11d42c7f6bc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:04:29.595058 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 781/1555] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 11:04:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 11:04:29.595995 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12575963-d68a-4aa1-9b60-11d42c7f6bc9 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:04:29.603356 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-901c24f8-5dc9-4930-bdac-bc4335c1eaac None None] MFA Rules not processed for user `ca155a48e2024c32af3fd2e19e5fac11`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 11:04:29.604578 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-901c24f8-5dc9-4930-bdac-bc4335c1eaac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:04:29.606198 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12575963-d68a-4aa1-9b60-11d42c7f6bc9 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:04:29.610742 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527c8002-559f-411d-8562-5885fce28d0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:04:29.611398 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527c8002-559f-411d-8562-5885fce28d0c admin admin] REQUEST_METHOD: `PUT` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:04:29.611672 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527c8002-559f-411d-8562-5885fce28d0c admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:04:29.611948 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527c8002-559f-411d-8562-5885fce28d0c admin admin] PATH_INFO: `/v3/projects/a3707cc107d343f9a1fd2df1ddc89441/users/8df99d3b39454d12a3742b84578dc818/roles/23e174ce743b46eabe618c212691da55` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:04:29.612534 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 775/1556] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 11:04:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 11:04:29.613757 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cff9ac-75b5-423a-9245-4ac89331bc33 None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:04:29.614065 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cff9ac-75b5-423a-9245-4ac89331bc33 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:04:29.614337 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cff9ac-75b5-423a-9245-4ac89331bc33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:04:29.635805 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-527c8002-559f-411d-8562-5885fce28d0c admin admin] RBAC: Authorizing `identity:create_grant(project_id=a3707cc107d343f9a1fd2df1ddc89441, user_id=8df99d3b39454d12a3742b84578dc818, role_id=23e174ce743b46eabe618c212691da55)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:04:29.635986 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-13cff9ac-75b5-423a-9245-4ac89331bc33 None None] MFA Rules not processed for user `ca155a48e2024c32af3fd2e19e5fac11`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 11:04:29.636777 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: WARNING py.warnings [None req-527c8002-559f-411d-8562-5885fce28d0c admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 26 11:04:29.636777 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: warnings.warn(msg) Aug 26 11:04:29.636777 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: Aug 26 11:04:29.637093 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-13cff9ac-75b5-423a-9245-4ac89331bc33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:04:29.638772 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-527c8002-559f-411d-8562-5885fce28d0c admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:04:29.645235 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 776/1557] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 11:04:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 11:04:29.646153 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13cff9ac-75b5-423a-9245-4ac89331bc33 None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:04:29.655422 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 782/1558] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 26 11:04:29 2022] PUT /identity/v3/projects/a3707cc107d343f9a1fd2df1ddc89441/users/8df99d3b39454d12a3742b84578dc818/roles/23e174ce743b46eabe618c212691da55 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 11:04:29.656347 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13cff9ac-75b5-423a-9245-4ac89331bc33 None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:04:29.656635 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e7cf4f-1238-416f-8831-11b9afb88641 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:04:29.656915 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e7cf4f-1238-416f-8831-11b9afb88641 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:04:29.657183 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e7cf4f-1238-416f-8831-11b9afb88641 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:04:29.677791 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-42e7cf4f-1238-416f-8831-11b9afb88641 None None] MFA Rules not processed for user `ca155a48e2024c32af3fd2e19e5fac11`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 11:04:29.679059 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-42e7cf4f-1238-416f-8831-11b9afb88641 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:04:29.716247 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6ccec3-1b1f-40df-a779-9bd5744f5018 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:04:29.716843 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6ccec3-1b1f-40df-a779-9bd5744f5018 admin admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:04:29.717120 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6ccec3-1b1f-40df-a779-9bd5744f5018 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:04:29.717392 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6ccec3-1b1f-40df-a779-9bd5744f5018 admin admin] PATH_INFO: `/v3/domains` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:04:29.718061 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d6ccec3-1b1f-40df-a779-9bd5744f5018 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 11:04:29.718361 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d6ccec3-1b1f-40df-a779-9bd5744f5018 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:04:29.719947 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d6ccec3-1b1f-40df-a779-9bd5744f5018 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:04:29.728552 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 11:04:29.732880 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 777/1559] 10.222.0.40 () {58 vars in 1153 bytes} [Fri Aug 26 11:04:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 11:04:29.734238 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6ccec3-1b1f-40df-a779-9bd5744f5018 admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:04:29.745577 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 783/1560] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 11:04:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 11:04:29.745577 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b37c10-06ac-462b-86c6-d55e55af44f4 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:04:29.745577 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b37c10-06ac-462b-86c6-d55e55af44f4 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:04:29.745577 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b37c10-06ac-462b-86c6-d55e55af44f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:04:29.748671 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6ccec3-1b1f-40df-a779-9bd5744f5018 admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:04:29.753426 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc12fb9-3f86-4eb5-b9c5-4f6bc71e3772 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:04:29.754043 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc12fb9-3f86-4eb5-b9c5-4f6bc71e3772 admin admin] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:04:29.754321 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc12fb9-3f86-4eb5-b9c5-4f6bc71e3772 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:04:29.754709 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc12fb9-3f86-4eb5-b9c5-4f6bc71e3772 admin admin] PATH_INFO: `/v3/projects` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:04:29.755355 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc12fb9-3f86-4eb5-b9c5-4f6bc71e3772 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:04:29.757758 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc12fb9-3f86-4eb5-b9c5-4f6bc71e3772 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:04:29.764617 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-74b37c10-06ac-462b-86c6-d55e55af44f4 None None] MFA Rules not processed for user `ca155a48e2024c32af3fd2e19e5fac11`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 11:04:29.765835 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-74b37c10-06ac-462b-86c6-d55e55af44f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:04:29.775421 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 784/1561] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 11:04:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 11:04:29.776977 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b37c10-06ac-462b-86c6-d55e55af44f4 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:04:29.781408 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 11:04:29.787505 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b37c10-06ac-462b-86c6-d55e55af44f4 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:04:29.795954 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 778/1562] 10.222.0.40 () {60 vars in 1153 bytes} [Fri Aug 26 11:04:29 2022] POST /identity/v3/projects => generated 379 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 11:04:29.796132 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43dae78d-eebd-48ed-b879-75ae9487b8d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:04:29.796132 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43dae78d-eebd-48ed-b879-75ae9487b8d0 admin admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:04:29.796319 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca8b2ae-d81d-4f21-a769-813add2add01 None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:04:29.796407 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43dae78d-eebd-48ed-b879-75ae9487b8d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:04:29.796490 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca8b2ae-d81d-4f21-a769-813add2add01 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:04:29.796582 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43dae78d-eebd-48ed-b879-75ae9487b8d0 admin admin] PATH_INFO: `/v3/domains` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:04:29.796669 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca8b2ae-d81d-4f21-a769-813add2add01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:04:29.796744 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43dae78d-eebd-48ed-b879-75ae9487b8d0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 11:04:29.796744 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43dae78d-eebd-48ed-b879-75ae9487b8d0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:04:29.797215 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43dae78d-eebd-48ed-b879-75ae9487b8d0 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:04:29.812995 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 785/1563] 10.222.0.40 () {58 vars in 1153 bytes} [Fri Aug 26 11:04:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 11:04:29.812995 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43dae78d-eebd-48ed-b879-75ae9487b8d0 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:04:29.813822 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-5ca8b2ae-d81d-4f21-a769-813add2add01 None None] MFA Rules not processed for user `ca155a48e2024c32af3fd2e19e5fac11`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 11:04:29.815041 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-5ca8b2ae-d81d-4f21-a769-813add2add01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:04:29.819410 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43dae78d-eebd-48ed-b879-75ae9487b8d0 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:04:29.890175 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa8d356-38e3-45d3-aafe-e7ed7a977891 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:04:29.890808 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa8d356-38e3-45d3-aafe-e7ed7a977891 admin admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:04:29.890988 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa8d356-38e3-45d3-aafe-e7ed7a977891 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:04:29.891258 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa8d356-38e3-45d3-aafe-e7ed7a977891 admin admin] PATH_INFO: `/v3/domains` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:04:29.892036 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa8d356-38e3-45d3-aafe-e7ed7a977891 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 11:04:29.892337 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa8d356-38e3-45d3-aafe-e7ed7a977891 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:04:29.893954 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa8d356-38e3-45d3-aafe-e7ed7a977891 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:04:29.895010 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 779/1564] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 11:04:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 11:04:29.896317 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14a2a4d-55a1-4832-a7f1-9769ac677373 None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:04:29.896629 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14a2a4d-55a1-4832-a7f1-9769ac677373 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:04:29.896875 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14a2a4d-55a1-4832-a7f1-9769ac677373 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:04:29.913761 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 786/1565] 10.222.0.40 () {58 vars in 1153 bytes} [Fri Aug 26 11:04:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 11:04:29.913761 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa8d356-38e3-45d3-aafe-e7ed7a977891 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:04:29.921558 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa8d356-38e3-45d3-aafe-e7ed7a977891 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:04:29.926415 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6f1dad-8b11-4d54-9818-644ab675800c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:04:29.927005 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6f1dad-8b11-4d54-9818-644ab675800c admin admin] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:04:29.927284 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6f1dad-8b11-4d54-9818-644ab675800c admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:04:29.927557 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6f1dad-8b11-4d54-9818-644ab675800c admin admin] PATH_INFO: `/v3/projects` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:04:29.928171 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab6f1dad-8b11-4d54-9818-644ab675800c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:04:29.928290 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-a14a2a4d-55a1-4832-a7f1-9769ac677373 None None] MFA Rules not processed for user `8df99d3b39454d12a3742b84578dc818`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 11:04:29.929753 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: WARNING py.warnings [None req-ab6f1dad-8b11-4d54-9818-644ab675800c admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 26 11:04:29.929753 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: warnings.warn(msg) Aug 26 11:04:29.929753 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: Aug 26 11:04:29.930108 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-a14a2a4d-55a1-4832-a7f1-9769ac677373 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:04:29.930956 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab6f1dad-8b11-4d54-9818-644ab675800c admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:04:29.961777 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 787/1566] 10.222.0.40 () {60 vars in 1153 bytes} [Fri Aug 26 11:04:29 2022] POST /identity/v3/projects => generated 391 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 11:04:29.961777 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6f1dad-8b11-4d54-9818-644ab675800c admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:04:29.999425 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 780/1567] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 11:04:29 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 11:04:30.000808 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4836b12c-cc1f-4c9f-bad2-8f5cc61d4a89 None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:04:30.001031 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4836b12c-cc1f-4c9f-bad2-8f5cc61d4a89 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:04:30.001297 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4836b12c-cc1f-4c9f-bad2-8f5cc61d4a89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:04:30.021760 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-4836b12c-cc1f-4c9f-bad2-8f5cc61d4a89 None None] MFA Rules not processed for user `ca155a48e2024c32af3fd2e19e5fac11`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 11:04:30.022638 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-4836b12c-cc1f-4c9f-bad2-8f5cc61d4a89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:04:30.025670 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6f1dad-8b11-4d54-9818-644ab675800c admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:04:30.030605 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3acc23-2c10-4a16-b8f5-6959c703315f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:04:30.031208 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3acc23-2c10-4a16-b8f5-6959c703315f admin admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:04:30.031574 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3acc23-2c10-4a16-b8f5-6959c703315f admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:04:30.031846 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3acc23-2c10-4a16-b8f5-6959c703315f admin admin] PATH_INFO: `/v3/domains` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:04:30.032609 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d3acc23-2c10-4a16-b8f5-6959c703315f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 11:04:30.032994 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d3acc23-2c10-4a16-b8f5-6959c703315f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:04:30.034469 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d3acc23-2c10-4a16-b8f5-6959c703315f admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:04:30.035230 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 781/1568] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 11:04:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 11:04:30.035981 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4836b12c-cc1f-4c9f-bad2-8f5cc61d4a89 None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:04:30.046662 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4836b12c-cc1f-4c9f-bad2-8f5cc61d4a89 None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:04:30.048319 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 788/1569] 10.222.0.40 () {58 vars in 1153 bytes} [Fri Aug 26 11:04:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 11:04:30.049047 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3acc23-2c10-4a16-b8f5-6959c703315f admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:04:30.054590 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecfcc2a1-d3bc-4245-82b8-1ac12b4b090f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:04:30.055208 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfcc2a1-d3bc-4245-82b8-1ac12b4b090f admin admin] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:04:30.055491 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfcc2a1-d3bc-4245-82b8-1ac12b4b090f admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:04:30.055846 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfcc2a1-d3bc-4245-82b8-1ac12b4b090f admin admin] PATH_INFO: `/v3/users` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:04:30.056484 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecfcc2a1-d3bc-4245-82b8-1ac12b4b090f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:04:30.057751 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: WARNING py.warnings [None req-ecfcc2a1-d3bc-4245-82b8-1ac12b4b090f admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 26 11:04:30.057751 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: warnings.warn(msg) Aug 26 11:04:30.057751 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: Aug 26 11:04:30.059434 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecfcc2a1-d3bc-4245-82b8-1ac12b4b090f admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:04:30.062405 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3acc23-2c10-4a16-b8f5-6959c703315f admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:04:30.066998 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbbd30db-14c7-4fd1-b3bc-5c1d76739a39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:04:30.067585 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbd30db-14c7-4fd1-b3bc-5c1d76739a39 admin admin] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:04:30.067860 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbd30db-14c7-4fd1-b3bc-5c1d76739a39 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:04:30.068127 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbd30db-14c7-4fd1-b3bc-5c1d76739a39 admin admin] PATH_INFO: `/v3/projects` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:04:30.068742 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbbd30db-14c7-4fd1-b3bc-5c1d76739a39 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:04:30.072375 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbbd30db-14c7-4fd1-b3bc-5c1d76739a39 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:04:30.086070 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 782/1570] 10.222.0.40 () {60 vars in 1144 bytes} [Fri Aug 26 11:04:30 2022] POST /identity/v3/users => generated 290 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 11:04:30.087189 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecfcc2a1-d3bc-4245-82b8-1ac12b4b090f admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:04:30.101542 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecfcc2a1-d3bc-4245-82b8-1ac12b4b090f admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:04:30.107678 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 789/1571] 10.222.0.40 () {60 vars in 1153 bytes} [Fri Aug 26 11:04:30 2022] POST /identity/v3/projects => generated 373 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 11:04:30.107678 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbbd30db-14c7-4fd1-b3bc-5c1d76739a39 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:04:30.107952 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b6f3db-4525-4e54-ab60-0fae4ec5e956 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:04:30.107952 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b6f3db-4525-4e54-ab60-0fae4ec5e956 admin admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:04:30.107952 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b6f3db-4525-4e54-ab60-0fae4ec5e956 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:04:30.107952 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b6f3db-4525-4e54-ab60-0fae4ec5e956 admin admin] PATH_INFO: `/v3/domains` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:04:30.108675 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b6f3db-4525-4e54-ab60-0fae4ec5e956 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 11:04:30.109001 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b6f3db-4525-4e54-ab60-0fae4ec5e956 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:04:30.110667 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b6f3db-4525-4e54-ab60-0fae4ec5e956 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:04:30.123300 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 783/1572] 10.222.0.40 () {58 vars in 1153 bytes} [Fri Aug 26 11:04:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 11:04:30.124075 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b6f3db-4525-4e54-ab60-0fae4ec5e956 admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:04:30.135416 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b6f3db-4525-4e54-ab60-0fae4ec5e956 admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:04:30.140076 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 11:04:30.167287 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 11:04:30.170258 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbbd30db-14c7-4fd1-b3bc-5c1d76739a39 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:04:30.175226 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c734f6b-6591-4472-bd6b-e9f5d044048b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:04:30.175837 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c734f6b-6591-4472-bd6b-e9f5d044048b admin admin] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:04:30.176113 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c734f6b-6591-4472-bd6b-e9f5d044048b admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:04:30.176383 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c734f6b-6591-4472-bd6b-e9f5d044048b admin admin] PATH_INFO: `/v3/projects` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:04:30.177001 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c734f6b-6591-4472-bd6b-e9f5d044048b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:04:30.178945 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c734f6b-6591-4472-bd6b-e9f5d044048b admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:04:30.198888 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e298baf3-16d3-47f3-acd4-53210b7997d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:04:30.199420 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e298baf3-16d3-47f3-acd4-53210b7997d1 admin admin] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:04:30.199745 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e298baf3-16d3-47f3-acd4-53210b7997d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:04:30.200044 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e298baf3-16d3-47f3-acd4-53210b7997d1 admin admin] PATH_INFO: `/v3/users` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:04:30.200863 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e298baf3-16d3-47f3-acd4-53210b7997d1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:04:30.202550 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e298baf3-16d3-47f3-acd4-53210b7997d1 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:04:30.209861 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 790/1573] 10.222.0.40 () {60 vars in 1153 bytes} [Fri Aug 26 11:04:30 2022] POST /identity/v3/projects => generated 377 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 11:04:30.209861 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad11d35-19b1-46b1-835f-11c24da86951 None None] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:04:30.209861 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad11d35-19b1-46b1-835f-11c24da86951 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:04:30.209861 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad11d35-19b1-46b1-835f-11c24da86951 None None] PATH_INFO: `/` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:04:30.209861 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 791/1574] 10.222.0.40 () {54 vars in 962 bytes} [Fri Aug 26 11:04:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 11:04:30.211166 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad11d35-19b1-46b1-835f-11c24da86951 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:04:30.230350 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 784/1575] 10.222.0.40 () {60 vars in 1144 bytes} [Fri Aug 26 11:04:30 2022] POST /identity/v3/users => generated 296 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 11:04:30.231155 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e298baf3-16d3-47f3-acd4-53210b7997d1 admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:04:30.240616 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e298baf3-16d3-47f3-acd4-53210b7997d1 admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:04:30.281063 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad11d35-19b1-46b1-835f-11c24da86951 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:04:30.286226 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711d46bf-b0d9-4607-89e1-7cb16d1ccb37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:04:30.286840 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711d46bf-b0d9-4607-89e1-7cb16d1ccb37 admin admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:04:30.287117 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711d46bf-b0d9-4607-89e1-7cb16d1ccb37 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:04:30.287412 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711d46bf-b0d9-4607-89e1-7cb16d1ccb37 admin admin] PATH_INFO: `/v3/domains` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:04:30.288060 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-711d46bf-b0d9-4607-89e1-7cb16d1ccb37 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 11:04:30.288385 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-711d46bf-b0d9-4607-89e1-7cb16d1ccb37 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:04:30.290065 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-711d46bf-b0d9-4607-89e1-7cb16d1ccb37 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:04:30.301366 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328cdd9a-30c6-4846-ba08-8d2850e2b6ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:04:30.301968 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328cdd9a-30c6-4846-ba08-8d2850e2b6ce admin admin] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:04:30.302298 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328cdd9a-30c6-4846-ba08-8d2850e2b6ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:04:30.302611 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 792/1576] 10.222.0.40 () {58 vars in 1153 bytes} [Fri Aug 26 11:04:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 11:04:30.302738 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328cdd9a-30c6-4846-ba08-8d2850e2b6ce admin admin] PATH_INFO: `/v3/projects` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:04:30.303462 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-328cdd9a-30c6-4846-ba08-8d2850e2b6ce admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:04:30.303581 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711d46bf-b0d9-4607-89e1-7cb16d1ccb37 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:04:30.305488 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-328cdd9a-30c6-4846-ba08-8d2850e2b6ce admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:04:30.314103 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711d46bf-b0d9-4607-89e1-7cb16d1ccb37 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:04:30.320214 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66bc477-dff2-4b23-9221-4262b629cdc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:04:30.320977 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66bc477-dff2-4b23-9221-4262b629cdc6 admin admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:04:30.321487 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66bc477-dff2-4b23-9221-4262b629cdc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:04:30.321763 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66bc477-dff2-4b23-9221-4262b629cdc6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:04:30.322439 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b66bc477-dff2-4b23-9221-4262b629cdc6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:04:30.324198 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b66bc477-dff2-4b23-9221-4262b629cdc6 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:04:30.333365 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 785/1577] 10.222.0.40 () {60 vars in 1153 bytes} [Fri Aug 26 11:04:30 2022] POST /identity/v3/projects => generated 389 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 11:04:30.333962 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328cdd9a-30c6-4846-ba08-8d2850e2b6ce admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:04:30.334603 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 793/1578] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 26 11:04:30 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 11:04:30.336038 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66bc477-dff2-4b23-9221-4262b629cdc6 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:04:30.346197 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66bc477-dff2-4b23-9221-4262b629cdc6 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:04:30.403329 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530c3127-d537-4e3e-81dc-f2c9a4c75378 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:04:30.403941 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530c3127-d537-4e3e-81dc-f2c9a4c75378 admin admin] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:04:30.404224 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530c3127-d537-4e3e-81dc-f2c9a4c75378 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:04:30.404496 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530c3127-d537-4e3e-81dc-f2c9a4c75378 admin admin] PATH_INFO: `/v3/projects` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:04:30.405075 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328cdd9a-30c6-4846-ba08-8d2850e2b6ce admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:04:30.405212 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-530c3127-d537-4e3e-81dc-f2c9a4c75378 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:04:30.407054 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-530c3127-d537-4e3e-81dc-f2c9a4c75378 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:04:30.409681 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd4277e-1c1e-4bd7-be62-708f92750ef0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:04:30.410301 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd4277e-1c1e-4bd7-be62-708f92750ef0 admin admin] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:04:30.410578 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd4277e-1c1e-4bd7-be62-708f92750ef0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:04:30.410847 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd4277e-1c1e-4bd7-be62-708f92750ef0 admin admin] PATH_INFO: `/v3/users` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:04:30.411459 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd4277e-1c1e-4bd7-be62-708f92750ef0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:04:30.413350 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd4277e-1c1e-4bd7-be62-708f92750ef0 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:04:30.434778 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 794/1579] 10.222.0.40 () {60 vars in 1153 bytes} [Fri Aug 26 11:04:30 2022] POST /identity/v3/projects => generated 381 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 11:04:30.435272 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530c3127-d537-4e3e-81dc-f2c9a4c75378 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:04:30.438522 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 786/1580] 10.222.0.40 () {60 vars in 1144 bytes} [Fri Aug 26 11:04:30 2022] POST /identity/v3/users => generated 287 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 11:04:30.440119 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f490e18d-217e-4bb2-bc42-6b7db5eedd1f None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:04:30.440395 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f490e18d-217e-4bb2-bc42-6b7db5eedd1f None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:04:30.440671 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f490e18d-217e-4bb2-bc42-6b7db5eedd1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:04:30.468312 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 11:04:30.481063 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-f490e18d-217e-4bb2-bc42-6b7db5eedd1f None None] MFA Rules not processed for user `1f4b4a662db049ecacc865ecff10bdb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 11:04:30.482304 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-f490e18d-217e-4bb2-bc42-6b7db5eedd1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:04:30.500121 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 11:04:30.502731 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530c3127-d537-4e3e-81dc-f2c9a4c75378 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:04:30.508386 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb19a2cf-778f-4ca9-a6df-7d0d50d56a5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:04:30.509059 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb19a2cf-778f-4ca9-a6df-7d0d50d56a5e admin admin] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:04:30.509666 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb19a2cf-778f-4ca9-a6df-7d0d50d56a5e admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:04:30.509666 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb19a2cf-778f-4ca9-a6df-7d0d50d56a5e admin admin] PATH_INFO: `/v3/users` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:04:30.510243 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb19a2cf-778f-4ca9-a6df-7d0d50d56a5e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:04:30.512151 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb19a2cf-778f-4ca9-a6df-7d0d50d56a5e admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:04:30.535312 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 795/1581] 10.222.0.40 () {60 vars in 1144 bytes} [Fri Aug 26 11:04:30 2022] POST /identity/v3/users => generated 289 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 11:04:30.536414 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb19a2cf-778f-4ca9-a6df-7d0d50d56a5e admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:04:30.546717 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb19a2cf-778f-4ca9-a6df-7d0d50d56a5e admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:04:30.551260 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40aeb4b-01cd-411b-874e-037779fdc08c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:04:30.551894 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40aeb4b-01cd-411b-874e-037779fdc08c admin admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:04:30.552110 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40aeb4b-01cd-411b-874e-037779fdc08c admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:04:30.552381 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40aeb4b-01cd-411b-874e-037779fdc08c admin admin] PATH_INFO: `/v3/roles` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:04:30.553007 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f40aeb4b-01cd-411b-874e-037779fdc08c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:04:30.554656 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f40aeb4b-01cd-411b-874e-037779fdc08c admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:04:30.557802 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 787/1582] 10.222.0.40 () {58 vars in 1059 bytes} [Fri Aug 26 11:04:30 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 11:04:30.558970 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f490e18d-217e-4bb2-bc42-6b7db5eedd1f None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:04:30.569043 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f490e18d-217e-4bb2-bc42-6b7db5eedd1f None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:04:30.573882 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 796/1583] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 26 11:04:30 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 11:04:30.573882 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40aeb4b-01cd-411b-874e-037779fdc08c admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:04:30.577131 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19c4874-ee8b-43fd-9935-60c67037af77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:04:30.577726 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19c4874-ee8b-43fd-9935-60c67037af77 admin admin] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:04:30.578025 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19c4874-ee8b-43fd-9935-60c67037af77 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:04:30.578298 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19c4874-ee8b-43fd-9935-60c67037af77 admin admin] PATH_INFO: `/v3/projects` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:04:30.578917 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b19c4874-ee8b-43fd-9935-60c67037af77 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:04:30.580795 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b19c4874-ee8b-43fd-9935-60c67037af77 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:04:30.583528 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40aeb4b-01cd-411b-874e-037779fdc08c admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:04:30.588983 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7321b6ee-b029-4a18-a375-32646b7e3777 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:04:30.589565 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7321b6ee-b029-4a18-a375-32646b7e3777 admin admin] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:04:30.589845 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7321b6ee-b029-4a18-a375-32646b7e3777 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:04:30.590150 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7321b6ee-b029-4a18-a375-32646b7e3777 admin admin] PATH_INFO: `/v3/users` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:04:30.590767 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7321b6ee-b029-4a18-a375-32646b7e3777 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:04:30.592653 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7321b6ee-b029-4a18-a375-32646b7e3777 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:04:30.610684 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 788/1584] 10.222.0.40 () {60 vars in 1153 bytes} [Fri Aug 26 11:04:30 2022] POST /identity/v3/projects => generated 385 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 11:04:30.614222 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19c4874-ee8b-43fd-9935-60c67037af77 admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:04:30.617013 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 797/1585] 10.222.0.40 () {60 vars in 1144 bytes} [Fri Aug 26 11:04:30 2022] POST /identity/v3/users => generated 295 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 11:04:30.620121 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7321b6ee-b029-4a18-a375-32646b7e3777 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:04:30.628468 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7321b6ee-b029-4a18-a375-32646b7e3777 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:04:30.677924 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19c4874-ee8b-43fd-9935-60c67037af77 admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:04:30.683291 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c170cd-b9f1-45cb-9e88-8f194a2275e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:04:30.683888 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c170cd-b9f1-45cb-9e88-8f194a2275e8 admin admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:04:30.684169 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c170cd-b9f1-45cb-9e88-8f194a2275e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:04:30.684431 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c170cd-b9f1-45cb-9e88-8f194a2275e8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:04:30.685106 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c170cd-b9f1-45cb-9e88-8f194a2275e8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:04:30.686674 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c170cd-b9f1-45cb-9e88-8f194a2275e8 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:04:30.689788 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4298cf-0395-4b20-a77e-738ba004fdb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:04:30.690433 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4298cf-0395-4b20-a77e-738ba004fdb6 admin admin] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:04:30.690710 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4298cf-0395-4b20-a77e-738ba004fdb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:04:30.690981 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4298cf-0395-4b20-a77e-738ba004fdb6 admin admin] PATH_INFO: `/v3/users` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:04:30.691600 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc4298cf-0395-4b20-a77e-738ba004fdb6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:04:30.693499 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc4298cf-0395-4b20-a77e-738ba004fdb6 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:04:30.700581 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 789/1586] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 26 11:04:30 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 11:04:30.700581 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c170cd-b9f1-45cb-9e88-8f194a2275e8 admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:04:30.709440 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c170cd-b9f1-45cb-9e88-8f194a2275e8 admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:04:30.714586 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36335b93-ea1a-4fa3-a6aa-fa9d539d37fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:04:30.715174 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36335b93-ea1a-4fa3-a6aa-fa9d539d37fd admin admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:04:30.715448 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36335b93-ea1a-4fa3-a6aa-fa9d539d37fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:04:30.715715 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36335b93-ea1a-4fa3-a6aa-fa9d539d37fd admin admin] PATH_INFO: `/v3/roles` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:04:30.716300 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 798/1587] 10.222.0.40 () {60 vars in 1144 bytes} [Fri Aug 26 11:04:30 2022] POST /identity/v3/users => generated 291 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 11:04:30.716925 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4298cf-0395-4b20-a77e-738ba004fdb6 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:04:30.718278 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36335b93-ea1a-4fa3-a6aa-fa9d539d37fd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:04:30.720049 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36335b93-ea1a-4fa3-a6aa-fa9d539d37fd admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:04:30.726447 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4298cf-0395-4b20-a77e-738ba004fdb6 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:04:30.731148 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 790/1588] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 26 11:04:30 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 11:04:30.734627 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3554156a-e350-4bba-9faa-a4d183750a1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:04:30.734917 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36335b93-ea1a-4fa3-a6aa-fa9d539d37fd admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:04:30.735311 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3554156a-e350-4bba-9faa-a4d183750a1c admin admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:04:30.735549 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3554156a-e350-4bba-9faa-a4d183750a1c admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:04:30.735825 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3554156a-e350-4bba-9faa-a4d183750a1c admin admin] PATH_INFO: `/v3/roles` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:04:30.736497 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3554156a-e350-4bba-9faa-a4d183750a1c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:04:30.738126 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3554156a-e350-4bba-9faa-a4d183750a1c admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:04:30.752371 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 799/1589] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 26 11:04:30 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 11:04:30.752371 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3554156a-e350-4bba-9faa-a4d183750a1c admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:04:30.762611 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3554156a-e350-4bba-9faa-a4d183750a1c admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:04:30.767636 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa226a3-a665-4b14-8cec-3b65393af395 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:04:30.768309 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa226a3-a665-4b14-8cec-3b65393af395 admin admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:04:30.768753 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa226a3-a665-4b14-8cec-3b65393af395 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:04:30.769091 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa226a3-a665-4b14-8cec-3b65393af395 admin admin] PATH_INFO: `/v3/roles` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:04:30.769819 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa226a3-a665-4b14-8cec-3b65393af395 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:04:30.771537 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa226a3-a665-4b14-8cec-3b65393af395 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:04:30.786336 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 800/1590] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 26 11:04:30 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 11:04:30.786336 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa226a3-a665-4b14-8cec-3b65393af395 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:04:30.794092 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa226a3-a665-4b14-8cec-3b65393af395 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:04:30.798785 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4623347e-fea2-4b13-938f-3f18f365ab36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:04:30.799349 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4623347e-fea2-4b13-938f-3f18f365ab36 admin admin] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:04:30.799620 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4623347e-fea2-4b13-938f-3f18f365ab36 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:04:30.799886 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4623347e-fea2-4b13-938f-3f18f365ab36 admin admin] PATH_INFO: `/v3/users` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:04:30.800479 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4623347e-fea2-4b13-938f-3f18f365ab36 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:04:30.802389 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4623347e-fea2-4b13-938f-3f18f365ab36 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:04:30.809377 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36335b93-ea1a-4fa3-a6aa-fa9d539d37fd admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:04:30.815040 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a0b4d6-ab30-4f39-b0fe-138e82cf7c96 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f4b4a662db049ecacc865ecff10bdb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e30ccf0c1204727b786876c141dbcd2', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:04:30.815725 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a0b4d6-ab30-4f39-b0fe-138e82cf7c96 service nova] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:04:30.816003 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a0b4d6-ab30-4f39-b0fe-138e82cf7c96 service nova] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:04:30.816285 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a0b4d6-ab30-4f39-b0fe-138e82cf7c96 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:04:30.827803 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 801/1591] 10.222.0.40 () {60 vars in 1144 bytes} [Fri Aug 26 11:04:30 2022] POST /identity/v3/users => generated 293 bytes in 41 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 11:04:30.827803 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4623347e-fea2-4b13-938f-3f18f365ab36 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:04:30.832165 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 11:04:30.836362 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4623347e-fea2-4b13-938f-3f18f365ab36 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:04:30.841321 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6de543-2693-404c-b61b-11d2e9249581 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:04:30.841918 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6de543-2693-404c-b61b-11d2e9249581 admin admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:04:30.842228 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6de543-2693-404c-b61b-11d2e9249581 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:04:30.842551 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6de543-2693-404c-b61b-11d2e9249581 admin admin] PATH_INFO: `/v3/roles` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:04:30.843184 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc6de543-2693-404c-b61b-11d2e9249581 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:04:30.844766 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc6de543-2693-404c-b61b-11d2e9249581 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:04:30.855305 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 802/1592] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 26 11:04:30 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 11:04:30.856239 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6de543-2693-404c-b61b-11d2e9249581 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:04:30.867479 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6de543-2693-404c-b61b-11d2e9249581 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:04:30.872587 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c11c134-32dd-4f68-9bc2-fdcceb8e5cb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:04:30.873280 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c11c134-32dd-4f68-9bc2-fdcceb8e5cb5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:04:30.873560 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c11c134-32dd-4f68-9bc2-fdcceb8e5cb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:04:30.873828 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c11c134-32dd-4f68-9bc2-fdcceb8e5cb5 admin admin] PATH_INFO: `/v3/projects/6a00e426be1a41b4b37f7825869ac5cb/users/d074367ea693406badce6f7b468f3147/roles/23e174ce743b46eabe618c212691da55` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:04:30.883123 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a0b4d6-ab30-4f39-b0fe-138e82cf7c96 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:04:30.884738 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a0b4d6-ab30-4f39-b0fe-138e82cf7c96 service nova] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:04:30.890190 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c11c134-32dd-4f68-9bc2-fdcceb8e5cb5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6a00e426be1a41b4b37f7825869ac5cb, user_id=d074367ea693406badce6f7b468f3147, role_id=23e174ce743b46eabe618c212691da55)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:04:30.892820 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c11c134-32dd-4f68-9bc2-fdcceb8e5cb5 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:04:30.893851 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 791/1593] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 11:04:30 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 11:04:30.894674 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a0b4d6-ab30-4f39-b0fe-138e82cf7c96 service nova] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:04:30.905827 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a0b4d6-ab30-4f39-b0fe-138e82cf7c96 service nova] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:04:30.911250 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5c1ca5-fc7d-4033-b7b3-677dc1d500ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:04:30.911843 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5c1ca5-fc7d-4033-b7b3-677dc1d500ec admin admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:04:30.912120 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5c1ca5-fc7d-4033-b7b3-677dc1d500ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:04:30.912393 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5c1ca5-fc7d-4033-b7b3-677dc1d500ec admin admin] PATH_INFO: `/v3/roles` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:04:30.913029 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d5c1ca5-fc7d-4033-b7b3-677dc1d500ec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:04:30.915189 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d5c1ca5-fc7d-4033-b7b3-677dc1d500ec admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:04:30.917528 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 803/1594] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 26 11:04:30 2022] PUT /identity/v3/projects/6a00e426be1a41b4b37f7825869ac5cb/users/d074367ea693406badce6f7b468f3147/roles/23e174ce743b46eabe618c212691da55 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 11:04:30.918719 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c11c134-32dd-4f68-9bc2-fdcceb8e5cb5 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:04:30.925555 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=87,keystone:INSERT=13 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 11:04:30.929272 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 792/1595] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 26 11:04:30 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 11:04:30.929272 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5c1ca5-fc7d-4033-b7b3-677dc1d500ec admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:04:30.938163 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5c1ca5-fc7d-4033-b7b3-677dc1d500ec admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:04:30.997823 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c11c134-32dd-4f68-9bc2-fdcceb8e5cb5 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:04:31.002141 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a3db7d-0bd4-42a7-a638-d89761b7f3d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:04:31.002614 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3604bb-096f-4392-9760-b648a99a5947 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:04:31.003106 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3604bb-096f-4392-9760-b648a99a5947 admin admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:04:31.003386 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3604bb-096f-4392-9760-b648a99a5947 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:04:31.003672 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3604bb-096f-4392-9760-b648a99a5947 admin admin] PATH_INFO: `/v3/roles` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:04:31.004117 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a3db7d-0bd4-42a7-a638-d89761b7f3d6 admin admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:04:31.004290 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f3604bb-096f-4392-9760-b648a99a5947 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:04:31.004739 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a3db7d-0bd4-42a7-a638-d89761b7f3d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:04:31.005228 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a3db7d-0bd4-42a7-a638-d89761b7f3d6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:04:31.005915 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f3604bb-096f-4392-9760-b648a99a5947 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:04:31.006375 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a3db7d-0bd4-42a7-a638-d89761b7f3d6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:04:31.009062 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a3db7d-0bd4-42a7-a638-d89761b7f3d6 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:04:31.017027 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 804/1596] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 26 11:04:30 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 11:04:31.017886 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3604bb-096f-4392-9760-b648a99a5947 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:04:31.021590 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 793/1597] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 26 11:04:30 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 11:04:31.023916 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a3db7d-0bd4-42a7-a638-d89761b7f3d6 admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:04:31.030397 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3604bb-096f-4392-9760-b648a99a5947 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:04:31.035050 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1864d371-1083-4be8-8804-153908c7e513 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:04:31.035207 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a3db7d-0bd4-42a7-a638-d89761b7f3d6 admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:04:31.035699 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1864d371-1083-4be8-8804-153908c7e513 admin admin] REQUEST_METHOD: `PUT` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:04:31.035979 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1864d371-1083-4be8-8804-153908c7e513 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:04:31.036257 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1864d371-1083-4be8-8804-153908c7e513 admin admin] PATH_INFO: `/v3/projects/7db3b983562e49cfbb77c9e64b6e6cb9/users/d6dbbabdd88c4c088bc91e6c43531318/roles/23e174ce743b46eabe618c212691da55` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:04:31.041133 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f472d7-7bf8-415f-8268-ad45cb0003c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:04:31.041873 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f472d7-7bf8-415f-8268-ad45cb0003c6 admin admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:04:31.042338 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f472d7-7bf8-415f-8268-ad45cb0003c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:04:31.042667 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f472d7-7bf8-415f-8268-ad45cb0003c6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:04:31.043354 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f472d7-7bf8-415f-8268-ad45cb0003c6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:04:31.045151 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f472d7-7bf8-415f-8268-ad45cb0003c6 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:04:31.052232 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1864d371-1083-4be8-8804-153908c7e513 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7db3b983562e49cfbb77c9e64b6e6cb9, user_id=d6dbbabdd88c4c088bc91e6c43531318, role_id=23e174ce743b46eabe618c212691da55)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:04:31.055688 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1864d371-1083-4be8-8804-153908c7e513 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:04:31.071214 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 794/1598] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 26 11:04:31 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 11:04:31.071214 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f472d7-7bf8-415f-8268-ad45cb0003c6 admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:04:31.080332 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 805/1599] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 26 11:04:31 2022] PUT /identity/v3/projects/7db3b983562e49cfbb77c9e64b6e6cb9/users/d6dbbabdd88c4c088bc91e6c43531318/roles/23e174ce743b46eabe618c212691da55 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 11:04:31.080332 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19557b1e-2e37-4bdc-a092-ab94421047aa None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:04:31.080332 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19557b1e-2e37-4bdc-a092-ab94421047aa None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:04:31.080332 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19557b1e-2e37-4bdc-a092-ab94421047aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:04:31.082243 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f472d7-7bf8-415f-8268-ad45cb0003c6 admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:04:31.101589 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-19557b1e-2e37-4bdc-a092-ab94421047aa None None] MFA Rules not processed for user `ca155a48e2024c32af3fd2e19e5fac11`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 11:04:31.102904 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-19557b1e-2e37-4bdc-a092-ab94421047aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:04:31.144898 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6242b0dd-f652-41b9-a561-e940d603efd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:04:31.145506 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6242b0dd-f652-41b9-a561-e940d603efd7 admin admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:04:31.145779 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6242b0dd-f652-41b9-a561-e940d603efd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:04:31.146084 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6242b0dd-f652-41b9-a561-e940d603efd7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:04:31.146723 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6242b0dd-f652-41b9-a561-e940d603efd7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:04:31.149619 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6242b0dd-f652-41b9-a561-e940d603efd7 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:04:31.160005 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 795/1600] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 26 11:04:31 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 11:04:31.160930 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6242b0dd-f652-41b9-a561-e940d603efd7 admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:04:31.167767 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 806/1601] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 11:04:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 11:04:31.168083 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19557b1e-2e37-4bdc-a092-ab94421047aa None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:04:31.171249 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6242b0dd-f652-41b9-a561-e940d603efd7 admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:04:31.176346 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68673556-bd1a-46c6-b6b4-409e9915f8bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:04:31.176948 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68673556-bd1a-46c6-b6b4-409e9915f8bb admin admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:04:31.177223 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68673556-bd1a-46c6-b6b4-409e9915f8bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:04:31.177548 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68673556-bd1a-46c6-b6b4-409e9915f8bb admin admin] PATH_INFO: `/v3/roles` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:04:31.177947 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19557b1e-2e37-4bdc-a092-ab94421047aa None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:04:31.178223 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68673556-bd1a-46c6-b6b4-409e9915f8bb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:04:31.180242 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68673556-bd1a-46c6-b6b4-409e9915f8bb admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:04:31.182894 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b4fe12-9c24-4cb6-961f-758f5da8e50f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:04:31.183616 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b4fe12-9c24-4cb6-961f-758f5da8e50f admin admin] REQUEST_METHOD: `PUT` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:04:31.183969 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b4fe12-9c24-4cb6-961f-758f5da8e50f admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:04:31.184252 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b4fe12-9c24-4cb6-961f-758f5da8e50f admin admin] PATH_INFO: `/v3/projects/c6a71f4243b14f2aa2fe5c8b8e3741e1/users/e3201429fd7242d39007fb286cade424/roles/23e174ce743b46eabe618c212691da55` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:04:31.193569 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 796/1602] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 26 11:04:31 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 11:04:31.193569 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68673556-bd1a-46c6-b6b4-409e9915f8bb admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:04:31.200675 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b4fe12-9c24-4cb6-961f-758f5da8e50f admin admin] RBAC: Authorizing `identity:create_grant(project_id=c6a71f4243b14f2aa2fe5c8b8e3741e1, user_id=e3201429fd7242d39007fb286cade424, role_id=23e174ce743b46eabe618c212691da55)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:04:31.201587 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68673556-bd1a-46c6-b6b4-409e9915f8bb admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:04:31.203363 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b4fe12-9c24-4cb6-961f-758f5da8e50f admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:04:31.206587 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735a06e1-f273-448f-8610-038bcc1e6547 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:04:31.207223 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735a06e1-f273-448f-8610-038bcc1e6547 admin admin] REQUEST_METHOD: `PUT` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:04:31.207495 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735a06e1-f273-448f-8610-038bcc1e6547 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:04:31.207783 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735a06e1-f273-448f-8610-038bcc1e6547 admin admin] PATH_INFO: `/v3/projects/0fa496929354424ca7d024a8279f2e80/users/163e8be86d5640f88893c50e6f3a8af7/roles/23e174ce743b46eabe618c212691da55` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:04:31.217531 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 807/1603] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 26 11:04:31 2022] PUT /identity/v3/projects/c6a71f4243b14f2aa2fe5c8b8e3741e1/users/e3201429fd7242d39007fb286cade424/roles/23e174ce743b46eabe618c212691da55 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 11:04:31.218413 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b4fe12-9c24-4cb6-961f-758f5da8e50f admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:04:31.220070 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 11:04:31.225685 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-735a06e1-f273-448f-8610-038bcc1e6547 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0fa496929354424ca7d024a8279f2e80, user_id=163e8be86d5640f88893c50e6f3a8af7, role_id=23e174ce743b46eabe618c212691da55)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:04:31.229020 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-735a06e1-f273-448f-8610-038bcc1e6547 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:04:31.243614 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 797/1604] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 26 11:04:31 2022] PUT /identity/v3/projects/0fa496929354424ca7d024a8279f2e80/users/163e8be86d5640f88893c50e6f3a8af7/roles/23e174ce743b46eabe618c212691da55 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 11:04:31.247793 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdd02e6-2b00-4920-b435-c8db9732b476 None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:04:31.247793 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdd02e6-2b00-4920-b435-c8db9732b476 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:04:31.247793 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdd02e6-2b00-4920-b435-c8db9732b476 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:04:31.266344 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-ebdd02e6-2b00-4920-b435-c8db9732b476 None None] MFA Rules not processed for user `ca155a48e2024c32af3fd2e19e5fac11`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 11:04:31.267546 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-ebdd02e6-2b00-4920-b435-c8db9732b476 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:04:31.274449 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 11:04:31.288926 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b4fe12-9c24-4cb6-961f-758f5da8e50f admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:04:31.328286 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 798/1605] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 11:04:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 11:04:31.329539 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77511397-a051-49c0-a5d2-c09d4e8f2fc1 None None] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:04:31.329814 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77511397-a051-49c0-a5d2-c09d4e8f2fc1 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:04:31.330115 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77511397-a051-49c0-a5d2-c09d4e8f2fc1 None None] PATH_INFO: `/` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:04:31.331359 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 799/1606] 10.222.0.40 () {54 vars in 912 bytes} [Fri Aug 26 11:04:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 11:04:31.336236 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77511397-a051-49c0-a5d2-c09d4e8f2fc1 None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:04:31.348777 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77511397-a051-49c0-a5d2-c09d4e8f2fc1 None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:04:31.350069 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a62bd9f-9eb1-46aa-adb9-2beba06c26c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:04:31.350828 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a62bd9f-9eb1-46aa-adb9-2beba06c26c1 admin admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:04:31.351726 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a62bd9f-9eb1-46aa-adb9-2beba06c26c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:04:31.352102 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a62bd9f-9eb1-46aa-adb9-2beba06c26c1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:04:31.352815 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a62bd9f-9eb1-46aa-adb9-2beba06c26c1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:04:31.354047 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed5f57e-bdc4-4ab2-b3c0-b208ce22b25c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:04:31.354585 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a62bd9f-9eb1-46aa-adb9-2beba06c26c1 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:04:31.354707 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed5f57e-bdc4-4ab2-b3c0-b208ce22b25c admin admin] REQUEST_METHOD: `PUT` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:04:31.354946 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed5f57e-bdc4-4ab2-b3c0-b208ce22b25c admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:04:31.355216 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed5f57e-bdc4-4ab2-b3c0-b208ce22b25c admin admin] PATH_INFO: `/v3/projects/78251e5bdb2548d1bb18aacd4ccbcd34/users/aedac3f62dd94f87b968f6c71d68a0a3/roles/23e174ce743b46eabe618c212691da55` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:04:31.364420 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 808/1607] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 26 11:04:31 2022] GET /identity/v3/roles => generated 1420 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 11:04:31.365219 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a62bd9f-9eb1-46aa-adb9-2beba06c26c1 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:04:31.371820 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed5f57e-bdc4-4ab2-b3c0-b208ce22b25c admin admin] RBAC: Authorizing `identity:create_grant(project_id=78251e5bdb2548d1bb18aacd4ccbcd34, user_id=aedac3f62dd94f87b968f6c71d68a0a3, role_id=23e174ce743b46eabe618c212691da55)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:04:31.374691 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed5f57e-bdc4-4ab2-b3c0-b208ce22b25c admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:04:31.389145 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 800/1608] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 26 11:04:31 2022] PUT /identity/v3/projects/78251e5bdb2548d1bb18aacd4ccbcd34/users/aedac3f62dd94f87b968f6c71d68a0a3/roles/23e174ce743b46eabe618c212691da55 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 11:04:31.389954 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed5f57e-bdc4-4ab2-b3c0-b208ce22b25c admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:04:31.430707 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a62bd9f-9eb1-46aa-adb9-2beba06c26c1 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:04:31.456347 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed5f57e-bdc4-4ab2-b3c0-b208ce22b25c admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:04:31.461707 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bdd6e57-3482-4506-a3a6-7d613ed8e9b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:04:31.462398 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdd6e57-3482-4506-a3a6-7d613ed8e9b1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:04:31.462674 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdd6e57-3482-4506-a3a6-7d613ed8e9b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:04:31.462983 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdd6e57-3482-4506-a3a6-7d613ed8e9b1 admin admin] PATH_INFO: `/v3/projects/b37ac3b5e4114f02b08a4b2c8c3b60f8/users/f1e8970431174986a95dd75a605db777/roles/23e174ce743b46eabe618c212691da55` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:04:31.477881 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bdd6e57-3482-4506-a3a6-7d613ed8e9b1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b37ac3b5e4114f02b08a4b2c8c3b60f8, user_id=f1e8970431174986a95dd75a605db777, role_id=23e174ce743b46eabe618c212691da55)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:04:31.480122 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7f21cd-8d5d-496a-80da-93a6fb9b8737 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91da2acd2534423091537e1ba068d2b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e30ccf0c1204727b786876c141dbcd2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:04:31.480556 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bdd6e57-3482-4506-a3a6-7d613ed8e9b1 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:04:31.480734 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7f21cd-8d5d-496a-80da-93a6fb9b8737 service neutron] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:04:31.481015 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7f21cd-8d5d-496a-80da-93a6fb9b8737 service neutron] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:04:31.481292 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7f21cd-8d5d-496a-80da-93a6fb9b8737 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:04:31.492243 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e7f21cd-8d5d-496a-80da-93a6fb9b8737 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:04:31.493660 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e7f21cd-8d5d-496a-80da-93a6fb9b8737 service neutron] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:04:31.494742 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 801/1609] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 26 11:04:31 2022] PUT /identity/v3/projects/b37ac3b5e4114f02b08a4b2c8c3b60f8/users/f1e8970431174986a95dd75a605db777/roles/23e174ce743b46eabe618c212691da55 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 11:04:31.496039 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7bd6a4-5df0-49a5-a10c-dea6f97ba417 None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:04:31.496386 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7bd6a4-5df0-49a5-a10c-dea6f97ba417 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:04:31.496723 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7bd6a4-5df0-49a5-a10c-dea6f97ba417 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:04:31.503902 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 809/1610] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 11:04:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 11:04:31.505133 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f56f27-a35c-42fe-be92-179532600d4b None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:04:31.505421 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f56f27-a35c-42fe-be92-179532600d4b None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:04:31.505695 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f56f27-a35c-42fe-be92-179532600d4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:04:31.527052 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-33f56f27-a35c-42fe-be92-179532600d4b None None] MFA Rules not processed for user `ca155a48e2024c32af3fd2e19e5fac11`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 11:04:31.528482 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-33f56f27-a35c-42fe-be92-179532600d4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:04:31.529644 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-fc7bd6a4-5df0-49a5-a10c-dea6f97ba417 None None] MFA Rules not processed for user `e3201429fd7242d39007fb286cade424`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 11:04:31.530855 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-fc7bd6a4-5df0-49a5-a10c-dea6f97ba417 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:04:31.573654 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 11:04:31.585831 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 11:04:31.599581 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 810/1611] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 11:04:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 11:04:31.601603 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ddf70b9-3e57-4435-8385-ea67899fbdc2 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:04:31.602119 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ddf70b9-3e57-4435-8385-ea67899fbdc2 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:04:31.602855 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ddf70b9-3e57-4435-8385-ea67899fbdc2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:04:31.613947 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 802/1612] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 11:04:31 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 11:04:31.614801 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7bd6a4-5df0-49a5-a10c-dea6f97ba417 None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:04:31.616523 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-fc7bd6a4-5df0-49a5-a10c-dea6f97ba417 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:04:31.630679 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-1ddf70b9-3e57-4435-8385-ea67899fbdc2 None None] MFA Rules not processed for user `1f4b4a662db049ecacc865ecff10bdb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 11:04:31.632388 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-1ddf70b9-3e57-4435-8385-ea67899fbdc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:04:31.691428 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7bd6a4-5df0-49a5-a10c-dea6f97ba417 None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:04:31.695978 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0e40bf-97bb-417a-bedb-0d88f6041623 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91da2acd2534423091537e1ba068d2b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e30ccf0c1204727b786876c141dbcd2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:04:31.696877 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0e40bf-97bb-417a-bedb-0d88f6041623 service neutron] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:04:31.697168 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0e40bf-97bb-417a-bedb-0d88f6041623 service neutron] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:04:31.697440 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0e40bf-97bb-417a-bedb-0d88f6041623 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:04:31.712390 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e0e40bf-97bb-417a-bedb-0d88f6041623 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:04:31.714394 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e0e40bf-97bb-417a-bedb-0d88f6041623 service neutron] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:04:31.721032 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 811/1613] 10.222.0.40 () {58 vars in 1009 bytes} [Fri Aug 26 11:04:31 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 11:04:31.728488 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ddf70b9-3e57-4435-8385-ea67899fbdc2 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:04:31.728675 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 803/1614] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 11:04:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 11:04:31.730423 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223b9ba1-c1bd-4bea-b9c5-aee8e753a11b None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:04:31.730827 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223b9ba1-c1bd-4bea-b9c5-aee8e753a11b None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:04:31.731171 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223b9ba1-c1bd-4bea-b9c5-aee8e753a11b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:04:31.742471 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ddf70b9-3e57-4435-8385-ea67899fbdc2 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:04:31.748356 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f16e47a-7fa1-4821-8741-ce380ea9bf43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:04:31.749673 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f16e47a-7fa1-4821-8741-ce380ea9bf43 admin admin] REQUEST_METHOD: `PUT` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:04:31.750064 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f16e47a-7fa1-4821-8741-ce380ea9bf43 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:04:31.750469 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f16e47a-7fa1-4821-8741-ce380ea9bf43 admin admin] PATH_INFO: `/v3/projects/41e03cfe608f47beabce918524dfa2b6/users/bbeef9cfedfa4bd386617b92ba1dfc90/roles/23e174ce743b46eabe618c212691da55` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:04:31.755372 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-223b9ba1-c1bd-4bea-b9c5-aee8e753a11b None None] MFA Rules not processed for user `ca155a48e2024c32af3fd2e19e5fac11`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 11:04:31.758682 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-223b9ba1-c1bd-4bea-b9c5-aee8e753a11b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:04:31.771777 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 804/1615] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 11:04:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 11:04:31.774619 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd38d1a-e3a7-46a8-a854-93835673da8d None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:04:31.774967 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f16e47a-7fa1-4821-8741-ce380ea9bf43 admin admin] RBAC: Authorizing `identity:create_grant(project_id=41e03cfe608f47beabce918524dfa2b6, user_id=bbeef9cfedfa4bd386617b92ba1dfc90, role_id=23e174ce743b46eabe618c212691da55)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:04:31.775383 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd38d1a-e3a7-46a8-a854-93835673da8d None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:04:31.775861 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd38d1a-e3a7-46a8-a854-93835673da8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:04:31.777942 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f16e47a-7fa1-4821-8741-ce380ea9bf43 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:04:31.799155 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 812/1616] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 26 11:04:31 2022] PUT /identity/v3/projects/41e03cfe608f47beabce918524dfa2b6/users/bbeef9cfedfa4bd386617b92ba1dfc90/roles/23e174ce743b46eabe618c212691da55 => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 11:04:31.800084 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f16e47a-7fa1-4821-8741-ce380ea9bf43 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:04:31.836261 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-8fd38d1a-e3a7-46a8-a854-93835673da8d None None] MFA Rules not processed for user `f1e8970431174986a95dd75a605db777`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 11:04:31.839465 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-8fd38d1a-e3a7-46a8-a854-93835673da8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:04:31.899880 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f16e47a-7fa1-4821-8741-ce380ea9bf43 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:04:31.909567 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76233e26-16ae-4ddf-ad34-43edcab6b199 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91da2acd2534423091537e1ba068d2b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e30ccf0c1204727b786876c141dbcd2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:04:31.910572 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76233e26-16ae-4ddf-ad34-43edcab6b199 service neutron] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:04:31.910767 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76233e26-16ae-4ddf-ad34-43edcab6b199 service neutron] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:04:31.911242 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76233e26-16ae-4ddf-ad34-43edcab6b199 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:04:31.956818 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 805/1617] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 11:04:31 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 11:04:31.958843 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5136c7db-ee6b-46da-b340-7cba67248908 None None] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:04:31.959363 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5136c7db-ee6b-46da-b340-7cba67248908 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:04:31.959363 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5136c7db-ee6b-46da-b340-7cba67248908 None None] PATH_INFO: `/` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:04:31.963975 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 806/1618] 10.222.0.40 () {54 vars in 962 bytes} [Fri Aug 26 11:04:31 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 11:04:31.965280 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5136c7db-ee6b-46da-b340-7cba67248908 None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:04:31.986042 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76233e26-16ae-4ddf-ad34-43edcab6b199 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:04:31.987700 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76233e26-16ae-4ddf-ad34-43edcab6b199 service neutron] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:04:32.000974 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 813/1619] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 11:04:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 201 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 11:04:32.001995 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76233e26-16ae-4ddf-ad34-43edcab6b199 service neutron] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:04:32.013670 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76233e26-16ae-4ddf-ad34-43edcab6b199 service neutron] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:04:32.018955 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c012cfe3-a704-45fb-9e75-079af1212b31 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91da2acd2534423091537e1ba068d2b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e30ccf0c1204727b786876c141dbcd2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:04:32.019451 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c012cfe3-a704-45fb-9e75-079af1212b31 service neutron] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:04:32.019819 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c012cfe3-a704-45fb-9e75-079af1212b31 service neutron] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:04:32.020182 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c012cfe3-a704-45fb-9e75-079af1212b31 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:04:32.033118 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c012cfe3-a704-45fb-9e75-079af1212b31 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:04:32.034884 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5136c7db-ee6b-46da-b340-7cba67248908 None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:04:32.035359 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c012cfe3-a704-45fb-9e75-079af1212b31 service neutron] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:04:32.039013 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535829fc-f637-443f-949f-47de31e3ef72 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c277338a7f814cc6b01c5419775031bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e30ccf0c1204727b786876c141dbcd2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:04:32.039764 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535829fc-f637-443f-949f-47de31e3ef72 service cinder] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:04:32.040144 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535829fc-f637-443f-949f-47de31e3ef72 service cinder] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:04:32.040394 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 11:04:32.040618 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535829fc-f637-443f-949f-47de31e3ef72 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:04:32.050282 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 814/1620] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 11:04:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 11:04:32.051987 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e95703d-5c82-44b1-8866-441a06fdb197 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:04:32.053022 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e95703d-5c82-44b1-8866-441a06fdb197 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:04:32.053665 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e95703d-5c82-44b1-8866-441a06fdb197 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:04:32.067191 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 11:04:32.095302 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-5e95703d-5c82-44b1-8866-441a06fdb197 None None] MFA Rules not processed for user `bbeef9cfedfa4bd386617b92ba1dfc90`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 11:04:32.096533 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-5e95703d-5c82-44b1-8866-441a06fdb197 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:04:32.141491 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-535829fc-f637-443f-949f-47de31e3ef72 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:04:32.143159 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-535829fc-f637-443f-949f-47de31e3ef72 service cinder] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:04:32.175470 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 807/1621] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 11:04:31 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 210 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 11:04:32.178202 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535829fc-f637-443f-949f-47de31e3ef72 service cinder] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:04:32.186701 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 815/1622] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 11:04:32 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 11:04:32.188228 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b327ab3-82d2-4415-859c-9b3a912d11f2 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:04:32.188518 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b327ab3-82d2-4415-859c-9b3a912d11f2 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:04:32.188792 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b327ab3-82d2-4415-859c-9b3a912d11f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:04:32.207862 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535829fc-f637-443f-949f-47de31e3ef72 service cinder] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:04:32.212207 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-9b327ab3-82d2-4415-859c-9b3a912d11f2 None None] MFA Rules not processed for user `1f4b4a662db049ecacc865ecff10bdb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 11:04:32.213424 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-9b327ab3-82d2-4415-859c-9b3a912d11f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:04:32.223218 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b547ef32-975c-4cd2-9368-c8d4ca64a3ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:04:32.223835 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b547ef32-975c-4cd2-9368-c8d4ca64a3ff admin admin] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:04:32.224114 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b547ef32-975c-4cd2-9368-c8d4ca64a3ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:04:32.225080 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 816/1623] 10.222.0.40 () {58 vars in 1059 bytes} [Fri Aug 26 11:04:32 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 11:04:32.225427 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b547ef32-975c-4cd2-9368-c8d4ca64a3ff admin admin] PATH_INFO: `/v3/projects` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:04:32.230413 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b547ef32-975c-4cd2-9368-c8d4ca64a3ff admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:04:32.232393 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b547ef32-975c-4cd2-9368-c8d4ca64a3ff admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:04:32.234415 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b327ab3-82d2-4415-859c-9b3a912d11f2 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:04:32.250890 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b327ab3-82d2-4415-859c-9b3a912d11f2 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:04:32.259946 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae55cf6b-83e8-4e4e-8d06-5bf636dd6998 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f4b4a662db049ecacc865ecff10bdb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e30ccf0c1204727b786876c141dbcd2', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:04:32.261699 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae55cf6b-83e8-4e4e-8d06-5bf636dd6998 service nova] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:04:32.262220 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae55cf6b-83e8-4e4e-8d06-5bf636dd6998 service nova] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:04:32.262906 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae55cf6b-83e8-4e4e-8d06-5bf636dd6998 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:04:32.292808 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 808/1624] 10.222.0.40 () {60 vars in 1153 bytes} [Fri Aug 26 11:04:32 2022] POST /identity/v3/projects => generated 381 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 11:04:32.293738 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b547ef32-975c-4cd2-9368-c8d4ca64a3ff admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:04:32.356955 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae55cf6b-83e8-4e4e-8d06-5bf636dd6998 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:04:32.359406 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae55cf6b-83e8-4e4e-8d06-5bf636dd6998 service nova] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:04:32.392288 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 817/1625] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 11:04:32 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 11:04:32.432368 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b547ef32-975c-4cd2-9368-c8d4ca64a3ff admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:04:32.446990 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd5979e-9137-440e-b0b0-2e1f03464ea2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:04:32.447620 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd5979e-9137-440e-b0b0-2e1f03464ea2 admin admin] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:04:32.447899 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd5979e-9137-440e-b0b0-2e1f03464ea2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:04:32.448171 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd5979e-9137-440e-b0b0-2e1f03464ea2 admin admin] PATH_INFO: `/v3/users` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:04:32.448801 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd5979e-9137-440e-b0b0-2e1f03464ea2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:04:32.450740 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd5979e-9137-440e-b0b0-2e1f03464ea2 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:04:32.512908 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 809/1626] 10.222.0.40 () {60 vars in 1144 bytes} [Fri Aug 26 11:04:32 2022] POST /identity/v3/users => generated 297 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 11:04:32.516569 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae55cf6b-83e8-4e4e-8d06-5bf636dd6998 service nova] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:04:32.535143 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae55cf6b-83e8-4e4e-8d06-5bf636dd6998 service nova] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:04:32.552346 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8471a21-2f24-4232-a4e7-ee9dbcc9faa1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:04:32.553019 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8471a21-2f24-4232-a4e7-ee9dbcc9faa1 admin admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:04:32.553245 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8471a21-2f24-4232-a4e7-ee9dbcc9faa1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:04:32.553519 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8471a21-2f24-4232-a4e7-ee9dbcc9faa1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:04:32.554764 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8471a21-2f24-4232-a4e7-ee9dbcc9faa1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:04:32.556549 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8471a21-2f24-4232-a4e7-ee9dbcc9faa1 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:04:32.604033 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 818/1627] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 26 11:04:32 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 11:04:32.608721 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd5979e-9137-440e-b0b0-2e1f03464ea2 admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:04:32.628616 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd5979e-9137-440e-b0b0-2e1f03464ea2 admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:04:32.642182 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f2b261-0133-454e-b2d8-b730243e7900 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:04:32.643697 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f2b261-0133-454e-b2d8-b730243e7900 admin admin] REQUEST_METHOD: `PUT` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:04:32.644029 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f2b261-0133-454e-b2d8-b730243e7900 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:04:32.644473 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f2b261-0133-454e-b2d8-b730243e7900 admin admin] PATH_INFO: `/v3/projects/1268423c21a043d38b2400a1ee2deec8/users/e4404bb8f8b942618c3b2992bcbe3d71/roles/1f97e4602e2e4d6498913a81f9ae8d6a` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:04:32.673627 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f2b261-0133-454e-b2d8-b730243e7900 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1268423c21a043d38b2400a1ee2deec8, user_id=e4404bb8f8b942618c3b2992bcbe3d71, role_id=1f97e4602e2e4d6498913a81f9ae8d6a)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:04:32.694095 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f2b261-0133-454e-b2d8-b730243e7900 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:04:32.724785 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 810/1628] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 26 11:04:32 2022] PUT /identity/v3/projects/1268423c21a043d38b2400a1ee2deec8/users/e4404bb8f8b942618c3b2992bcbe3d71/roles/1f97e4602e2e4d6498913a81f9ae8d6a => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 11:04:32.729516 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8471a21-2f24-4232-a4e7-ee9dbcc9faa1 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:04:32.749157 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8471a21-2f24-4232-a4e7-ee9dbcc9faa1 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:04:32.903272 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a8a030-d38d-4073-825b-ac47189a3b43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:04:32.904261 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a8a030-d38d-4073-825b-ac47189a3b43 admin admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:04:32.904678 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a8a030-d38d-4073-825b-ac47189a3b43 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:04:32.906011 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a8a030-d38d-4073-825b-ac47189a3b43 admin admin] PATH_INFO: `/v3/roles` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:04:32.906923 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a8a030-d38d-4073-825b-ac47189a3b43 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:04:32.913605 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a8a030-d38d-4073-825b-ac47189a3b43 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:04:32.928359 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f2b261-0133-454e-b2d8-b730243e7900 admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:04:32.952690 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 819/1629] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 26 11:04:32 2022] GET /identity/v3/roles => generated 1420 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 11:04:32.954138 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a8a030-d38d-4073-825b-ac47189a3b43 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:04:32.964580 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 11:04:32.969135 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a8a030-d38d-4073-825b-ac47189a3b43 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:04:32.984709 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d99d5b2-54c0-43bd-ac80-18b520a058a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:04:32.985404 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d99d5b2-54c0-43bd-ac80-18b520a058a9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:04:32.985685 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d99d5b2-54c0-43bd-ac80-18b520a058a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:04:32.985963 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d99d5b2-54c0-43bd-ac80-18b520a058a9 admin admin] PATH_INFO: `/v3/projects/1268423c21a043d38b2400a1ee2deec8/users/e4404bb8f8b942618c3b2992bcbe3d71/roles/1f97e4602e2e4d6498913a81f9ae8d6a` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:04:32.993616 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d99d5b2-54c0-43bd-ac80-18b520a058a9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1268423c21a043d38b2400a1ee2deec8, user_id=e4404bb8f8b942618c3b2992bcbe3d71, role_id=1f97e4602e2e4d6498913a81f9ae8d6a)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:04:32.996358 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d99d5b2-54c0-43bd-ac80-18b520a058a9 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:04:33.052432 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 820/1630] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 26 11:04:32 2022] PUT /identity/v3/projects/1268423c21a043d38b2400a1ee2deec8/users/e4404bb8f8b942618c3b2992bcbe3d71/roles/1f97e4602e2e4d6498913a81f9ae8d6a => generated 0 bytes in 89 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 11:04:33.054495 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce339ac9-897d-41ef-9900-3d9c1880e468 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:04:33.055049 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce339ac9-897d-41ef-9900-3d9c1880e468 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:04:33.055517 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce339ac9-897d-41ef-9900-3d9c1880e468 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:04:33.066955 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 11:04:33.105125 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-ce339ac9-897d-41ef-9900-3d9c1880e468 None None] MFA Rules not processed for user `e4404bb8f8b942618c3b2992bcbe3d71`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 11:04:33.106410 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-ce339ac9-897d-41ef-9900-3d9c1880e468 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:04:33.113294 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f2b261-0133-454e-b2d8-b730243e7900 admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:04:33.254708 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe1a118-2cae-4166-ae47-6d3958567c27 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f4b4a662db049ecacc865ecff10bdb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e30ccf0c1204727b786876c141dbcd2', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:04:33.255333 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe1a118-2cae-4166-ae47-6d3958567c27 service nova] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:04:33.255609 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe1a118-2cae-4166-ae47-6d3958567c27 service nova] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:04:33.255881 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe1a118-2cae-4166-ae47-6d3958567c27 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:04:33.287023 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 821/1631] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 11:04:33 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 234 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 11:04:33.297734 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce339ac9-897d-41ef-9900-3d9c1880e468 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:04:33.406222 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe1a118-2cae-4166-ae47-6d3958567c27 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:04:33.408153 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe1a118-2cae-4166-ae47-6d3958567c27 service nova] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:04:33.420808 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce339ac9-897d-41ef-9900-3d9c1880e468 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:04:33.427247 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48c50c0-095b-486e-9505-86be72762f9d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c277338a7f814cc6b01c5419775031bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e30ccf0c1204727b786876c141dbcd2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:04:33.428228 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48c50c0-095b-486e-9505-86be72762f9d service cinder] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:04:33.428743 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48c50c0-095b-486e-9505-86be72762f9d service cinder] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:04:33.429743 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48c50c0-095b-486e-9505-86be72762f9d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:04:33.440854 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 811/1632] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 11:04:32 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 513 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 11:04:33.445137 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b48c50c0-095b-486e-9505-86be72762f9d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:04:33.445137 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b48c50c0-095b-486e-9505-86be72762f9d service cinder] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:04:33.466392 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 822/1633] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 11:04:33 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 169 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 26 11:04:33.509324 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793e727e-4b2e-48a9-bb7f-e5d35f050fdf None None] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:04:33.509856 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793e727e-4b2e-48a9-bb7f-e5d35f050fdf None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:04:33.510602 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793e727e-4b2e-48a9-bb7f-e5d35f050fdf None None] PATH_INFO: `/` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:04:33.512533 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 812/1634] 10.222.0.40 () {54 vars in 912 bytes} [Fri Aug 26 11:04:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 11:04:33.524016 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be1f4fe-b6b8-4f7c-b7a3-b7d222dc2723 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:04:33.524710 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be1f4fe-b6b8-4f7c-b7a3-b7d222dc2723 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:04:33.524710 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be1f4fe-b6b8-4f7c-b7a3-b7d222dc2723 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:04:33.556563 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-7be1f4fe-b6b8-4f7c-b7a3-b7d222dc2723 None None] MFA Rules not processed for user `1f4b4a662db049ecacc865ecff10bdb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 11:04:33.557414 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-7be1f4fe-b6b8-4f7c-b7a3-b7d222dc2723 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:04:33.584998 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 823/1635] 10.222.0.40 () {58 vars in 1009 bytes} [Fri Aug 26 11:04:33 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 62 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 11:04:33.595635 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-793e727e-4b2e-48a9-bb7f-e5d35f050fdf None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:04:33.610893 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-793e727e-4b2e-48a9-bb7f-e5d35f050fdf None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:04:33.624959 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83fb5149-12c9-458c-9041-c2a6dc4341bf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c277338a7f814cc6b01c5419775031bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e30ccf0c1204727b786876c141dbcd2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:04:33.625604 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fb5149-12c9-458c-9041-c2a6dc4341bf service cinder] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:04:33.625858 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fb5149-12c9-458c-9041-c2a6dc4341bf service cinder] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:04:33.626169 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fb5149-12c9-458c-9041-c2a6dc4341bf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:04:33.647060 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83fb5149-12c9-458c-9041-c2a6dc4341bf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:04:33.649679 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83fb5149-12c9-458c-9041-c2a6dc4341bf service cinder] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:04:33.678822 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be1f4fe-b6b8-4f7c-b7a3-b7d222dc2723 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:04:33.689505 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be1f4fe-b6b8-4f7c-b7a3-b7d222dc2723 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:04:33.693420 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 813/1636] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 11:04:33 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 11:04:33.696493 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2605cb3-8c37-4c21-be4d-aee84d53c742 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c277338a7f814cc6b01c5419775031bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e30ccf0c1204727b786876c141dbcd2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:04:33.697590 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2605cb3-8c37-4c21-be4d-aee84d53c742 service cinder] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:04:33.698082 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2605cb3-8c37-4c21-be4d-aee84d53c742 service cinder] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:04:33.698538 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2605cb3-8c37-4c21-be4d-aee84d53c742 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:04:33.813307 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2605cb3-8c37-4c21-be4d-aee84d53c742 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:04:33.815137 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2605cb3-8c37-4c21-be4d-aee84d53c742 service cinder] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:04:33.826184 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 824/1637] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 11:04:33 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 11:04:34.152050 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83fb5149-12c9-458c-9041-c2a6dc4341bf service cinder] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:04:34.333578 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83fb5149-12c9-458c-9041-c2a6dc4341bf service cinder] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:04:34.341938 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800bff00-3178-4e30-bbb3-36dd0cebd879 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:04:34.342678 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800bff00-3178-4e30-bbb3-36dd0cebd879 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:04:34.342957 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800bff00-3178-4e30-bbb3-36dd0cebd879 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:04:34.343227 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800bff00-3178-4e30-bbb3-36dd0cebd879 admin admin] PATH_INFO: `/v3/users/e3201429fd7242d39007fb286cade424` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:04:34.344401 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-800bff00-3178-4e30-bbb3-36dd0cebd879 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e3201429fd7242d39007fb286cade424)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:04:34.346685 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-800bff00-3178-4e30-bbb3-36dd0cebd879 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:04:34.379308 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 11:04:34.462688 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-800bff00-3178-4e30-bbb3-36dd0cebd879 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e3201429fd7242d39007fb286cade424'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:04:34.475389 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-800bff00-3178-4e30-bbb3-36dd0cebd879 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e3201429fd7242d39007fb286cade424'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:04:34.476487 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-800bff00-3178-4e30-bbb3-36dd0cebd879 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e3201429fd7242d39007fb286cade424'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:04:34.506427 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-800bff00-3178-4e30-bbb3-36dd0cebd879 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e3201429fd7242d39007fb286cade424'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:04:34.508086 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-800bff00-3178-4e30-bbb3-36dd0cebd879 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e3201429fd7242d39007fb286cade424'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:04:34.533639 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 814/1638] 10.222.0.40 () {58 vars in 1224 bytes} [Fri Aug 26 11:04:34 2022] DELETE /identity/v3/users/e3201429fd7242d39007fb286cade424 => generated 0 bytes in 382 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 11:04:34.538027 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9412591-60ad-4bb8-b586-09e1e69a7408 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:04:34.538834 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9412591-60ad-4bb8-b586-09e1e69a7408 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:04:34.539381 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9412591-60ad-4bb8-b586-09e1e69a7408 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:04:34.593281 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-d9412591-60ad-4bb8-b586-09e1e69a7408 None None] MFA Rules not processed for user `ca155a48e2024c32af3fd2e19e5fac11`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 11:04:34.596544 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-d9412591-60ad-4bb8-b586-09e1e69a7408 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:04:34.614726 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 11:04:34.793386 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 825/1639] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 11:04:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 256 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 11:04:34.841858 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800bff00-3178-4e30-bbb3-36dd0cebd879 admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:04:34.844534 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-800bff00-3178-4e30-bbb3-36dd0cebd879 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:04:34.983749 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800bff00-3178-4e30-bbb3-36dd0cebd879 admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:04:34.990526 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c87809-7939-4c56-8355-628f690cd2f7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91da2acd2534423091537e1ba068d2b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e30ccf0c1204727b786876c141dbcd2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:04:34.991969 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c87809-7939-4c56-8355-628f690cd2f7 service neutron] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:04:34.992469 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c87809-7939-4c56-8355-628f690cd2f7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:04:34.993016 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c87809-7939-4c56-8355-628f690cd2f7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:04:35.013940 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c87809-7939-4c56-8355-628f690cd2f7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:04:35.017512 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c87809-7939-4c56-8355-628f690cd2f7 service neutron] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:04:35.034963 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 815/1640] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 11:04:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 195 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 11:04:36.224905 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9412591-60ad-4bb8-b586-09e1e69a7408 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:04:36.228081 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-d9412591-60ad-4bb8-b586-09e1e69a7408 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:04:36.250609 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9412591-60ad-4bb8-b586-09e1e69a7408 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:04:36.258023 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65dd8005-db25-4cbb-a6bc-2a50e20affe8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:04:36.259287 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65dd8005-db25-4cbb-a6bc-2a50e20affe8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:04:36.260917 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65dd8005-db25-4cbb-a6bc-2a50e20affe8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:04:36.261513 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65dd8005-db25-4cbb-a6bc-2a50e20affe8 admin admin] PATH_INFO: `/v3/users/8df99d3b39454d12a3742b84578dc818` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:04:36.263879 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65dd8005-db25-4cbb-a6bc-2a50e20affe8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8df99d3b39454d12a3742b84578dc818)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:04:36.265267 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: WARNING py.warnings [None req-65dd8005-db25-4cbb-a6bc-2a50e20affe8 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 26 11:04:36.265267 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: warnings.warn(msg) Aug 26 11:04:36.265267 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: Aug 26 11:04:36.267287 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65dd8005-db25-4cbb-a6bc-2a50e20affe8 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:04:36.422508 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-65dd8005-db25-4cbb-a6bc-2a50e20affe8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8df99d3b39454d12a3742b84578dc818'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:04:36.424256 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-65dd8005-db25-4cbb-a6bc-2a50e20affe8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8df99d3b39454d12a3742b84578dc818'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:04:36.450920 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-65dd8005-db25-4cbb-a6bc-2a50e20affe8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8df99d3b39454d12a3742b84578dc818'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:04:36.451767 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-65dd8005-db25-4cbb-a6bc-2a50e20affe8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8df99d3b39454d12a3742b84578dc818'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:04:36.525818 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c87809-7939-4c56-8355-628f690cd2f7 service neutron] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:04:36.528273 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-e7c87809-7939-4c56-8355-628f690cd2f7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:04:36.559617 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-65dd8005-db25-4cbb-a6bc-2a50e20affe8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8df99d3b39454d12a3742b84578dc818'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:04:36.576000 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c87809-7939-4c56-8355-628f690cd2f7 service neutron] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:04:36.608223 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4db148a-fa0a-4487-80ac-d40bb0bf485f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:04:36.613854 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4db148a-fa0a-4487-80ac-d40bb0bf485f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:04:36.614907 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4db148a-fa0a-4487-80ac-d40bb0bf485f admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:04:36.615250 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4db148a-fa0a-4487-80ac-d40bb0bf485f admin admin] PATH_INFO: `/v3/projects/c6a71f4243b14f2aa2fe5c8b8e3741e1` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:04:36.618978 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4db148a-fa0a-4487-80ac-d40bb0bf485f admin admin] RBAC: Authorizing `identity:delete_project(project_id=c6a71f4243b14f2aa2fe5c8b8e3741e1)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:04:36.626047 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: WARNING py.warnings [None req-c4db148a-fa0a-4487-80ac-d40bb0bf485f admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 26 11:04:36.626047 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: warnings.warn(msg) Aug 26 11:04:36.626047 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: Aug 26 11:04:36.626296 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 826/1641] 10.222.0.40 () {58 vars in 1224 bytes} [Fri Aug 26 11:04:36 2022] DELETE /identity/v3/users/8df99d3b39454d12a3742b84578dc818 => generated 0 bytes in 398 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 11:04:36.626360 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4db148a-fa0a-4487-80ac-d40bb0bf485f admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:04:36.626643 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f78ec8b-a95f-4ea5-9529-7bc20b1365b9 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:04:36.626983 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f78ec8b-a95f-4ea5-9529-7bc20b1365b9 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:04:36.627250 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f78ec8b-a95f-4ea5-9529-7bc20b1365b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:04:36.731557 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-4f78ec8b-a95f-4ea5-9529-7bc20b1365b9 None None] MFA Rules not processed for user `ca155a48e2024c32af3fd2e19e5fac11`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 11:04:36.737797 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-4f78ec8b-a95f-4ea5-9529-7bc20b1365b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:04:36.865611 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-c4db148a-fa0a-4487-80ac-d40bb0bf485f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c6a71f4243b14f2aa2fe5c8b8e3741e1', 'request_id': 'req-c4db148a-fa0a-4487-80ac-d40bb0bf485f'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:04:36.903947 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-c4db148a-fa0a-4487-80ac-d40bb0bf485f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c6a71f4243b14f2aa2fe5c8b8e3741e1', 'request_id': 'req-c4db148a-fa0a-4487-80ac-d40bb0bf485f'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:04:36.926042 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-c4db148a-fa0a-4487-80ac-d40bb0bf485f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c6a71f4243b14f2aa2fe5c8b8e3741e1', 'request_id': 'req-c4db148a-fa0a-4487-80ac-d40bb0bf485f'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:04:36.973578 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-c4db148a-fa0a-4487-80ac-d40bb0bf485f admin admin] The token cache is being invalidate because project c6a71f4243b14f2aa2fe5c8b8e3741e1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81423) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 11:04:36.974024 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-c4db148a-fa0a-4487-80ac-d40bb0bf485f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:04:36.978721 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 827/1642] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 11:04:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 350 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 11:04:36.978906 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-c4db148a-fa0a-4487-80ac-d40bb0bf485f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:04:36.983008 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 816/1643] 10.222.0.40 () {58 vars in 1233 bytes} [Fri Aug 26 11:04:36 2022] DELETE /identity/v3/projects/c6a71f4243b14f2aa2fe5c8b8e3741e1 => generated 0 bytes in 458 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 26 11:04:36.988679 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650a1ed6-a973-4833-9f74-27cf0d73081f None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:04:36.988679 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650a1ed6-a973-4833-9f74-27cf0d73081f None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:04:36.988874 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650a1ed6-a973-4833-9f74-27cf0d73081f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:04:37.015227 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-650a1ed6-a973-4833-9f74-27cf0d73081f None None] MFA Rules not processed for user `ca155a48e2024c32af3fd2e19e5fac11`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 11:04:37.027661 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-650a1ed6-a973-4833-9f74-27cf0d73081f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:04:37.079347 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 11:04:37.131130 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4db148a-fa0a-4487-80ac-d40bb0bf485f admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:04:37.132872 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-c4db148a-fa0a-4487-80ac-d40bb0bf485f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:04:37.207965 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 11:04:37.257185 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 828/1644] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 11:04:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 269 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 11:04:37.258850 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be36236c-6194-4edd-874c-8ae0078310c0 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:04:37.259223 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be36236c-6194-4edd-874c-8ae0078310c0 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:04:37.260153 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be36236c-6194-4edd-874c-8ae0078310c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:04:37.313447 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-be36236c-6194-4edd-874c-8ae0078310c0 None None] MFA Rules not processed for user `ca155a48e2024c32af3fd2e19e5fac11`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 11:04:37.315863 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-be36236c-6194-4edd-874c-8ae0078310c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:04:37.342804 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 829/1645] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 11:04:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 11:04:37.347468 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be36236c-6194-4edd-874c-8ae0078310c0 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:04:37.375131 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4db148a-fa0a-4487-80ac-d40bb0bf485f admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:04:37.380260 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be36236c-6194-4edd-874c-8ae0078310c0 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:04:37.381319 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674e8e3f-d91e-4aa4-87df-a25cffd785ac service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91da2acd2534423091537e1ba068d2b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e30ccf0c1204727b786876c141dbcd2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:04:37.382704 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674e8e3f-d91e-4aa4-87df-a25cffd785ac service neutron] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:04:37.383473 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674e8e3f-d91e-4aa4-87df-a25cffd785ac service neutron] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:04:37.384056 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674e8e3f-d91e-4aa4-87df-a25cffd785ac service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:04:37.387164 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfdabc18-fd48-472f-bebb-f42392e4f423 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:04:37.387308 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-674e8e3f-d91e-4aa4-87df-a25cffd785ac service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:04:37.387939 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfdabc18-fd48-472f-bebb-f42392e4f423 admin admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:04:37.388927 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfdabc18-fd48-472f-bebb-f42392e4f423 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:04:37.390069 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfdabc18-fd48-472f-bebb-f42392e4f423 admin admin] PATH_INFO: `/v3/domains` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:04:37.390854 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfdabc18-fd48-472f-bebb-f42392e4f423 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 11:04:37.391240 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfdabc18-fd48-472f-bebb-f42392e4f423 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:04:37.393869 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfdabc18-fd48-472f-bebb-f42392e4f423 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:04:37.418021 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 830/1646] 10.222.0.40 () {58 vars in 1153 bytes} [Fri Aug 26 11:04:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 11:04:37.419427 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-674e8e3f-d91e-4aa4-87df-a25cffd785ac service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:04:37.421070 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-674e8e3f-d91e-4aa4-87df-a25cffd785ac service neutron] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:04:37.422972 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfdabc18-fd48-472f-bebb-f42392e4f423 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:04:37.449699 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfdabc18-fd48-472f-bebb-f42392e4f423 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:04:37.455865 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 817/1647] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 11:04:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 11:04:37.460245 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f45ec4e-75c6-48c3-b63e-928d0c453f00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:04:37.461237 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f45ec4e-75c6-48c3-b63e-928d0c453f00 admin admin] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:04:37.461655 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f45ec4e-75c6-48c3-b63e-928d0c453f00 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:04:37.462144 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f45ec4e-75c6-48c3-b63e-928d0c453f00 admin admin] PATH_INFO: `/v3/projects` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:04:37.463327 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f45ec4e-75c6-48c3-b63e-928d0c453f00 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:04:37.465625 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f45ec4e-75c6-48c3-b63e-928d0c453f00 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:04:37.516322 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 831/1648] 10.222.0.40 () {60 vars in 1153 bytes} [Fri Aug 26 11:04:37 2022] POST /identity/v3/projects => generated 385 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 11:04:37.517934 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674e8e3f-d91e-4aa4-87df-a25cffd785ac service neutron] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:04:37.532550 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674e8e3f-d91e-4aa4-87df-a25cffd785ac service neutron] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:04:37.657147 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058f9b81-897a-416b-aa7d-a7da6af624d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:04:37.658357 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058f9b81-897a-416b-aa7d-a7da6af624d8 admin admin] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:04:37.658924 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058f9b81-897a-416b-aa7d-a7da6af624d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:04:37.659413 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058f9b81-897a-416b-aa7d-a7da6af624d8 admin admin] PATH_INFO: `/v3/users` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:04:37.660621 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-058f9b81-897a-416b-aa7d-a7da6af624d8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:04:37.664215 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-058f9b81-897a-416b-aa7d-a7da6af624d8 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:04:37.711672 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f45ec4e-75c6-48c3-b63e-928d0c453f00 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:04:37.712638 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 818/1649] 10.222.0.40 () {60 vars in 1144 bytes} [Fri Aug 26 11:04:37 2022] POST /identity/v3/users => generated 293 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 11:04:37.746555 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f45ec4e-75c6-48c3-b63e-928d0c453f00 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:04:37.755514 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ddf94e-78c6-4694-b9e1-745a48af87c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:04:37.756474 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ddf94e-78c6-4694-b9e1-745a48af87c6 admin admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:04:37.756906 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ddf94e-78c6-4694-b9e1-745a48af87c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:04:37.758214 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ddf94e-78c6-4694-b9e1-745a48af87c6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:04:37.759601 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ddf94e-78c6-4694-b9e1-745a48af87c6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:04:37.762234 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ddf94e-78c6-4694-b9e1-745a48af87c6 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:04:37.805782 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 832/1650] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 26 11:04:37 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 11:04:37.809788 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058f9b81-897a-416b-aa7d-a7da6af624d8 admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:04:37.826899 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058f9b81-897a-416b-aa7d-a7da6af624d8 admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:04:37.838737 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbefdd75-8114-4c90-850e-23cb42a14c9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:04:37.839943 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbefdd75-8114-4c90-850e-23cb42a14c9e admin admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:04:37.840239 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbefdd75-8114-4c90-850e-23cb42a14c9e admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:04:37.840513 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbefdd75-8114-4c90-850e-23cb42a14c9e admin admin] PATH_INFO: `/v3/roles` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:04:37.841170 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbefdd75-8114-4c90-850e-23cb42a14c9e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:04:37.842813 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbefdd75-8114-4c90-850e-23cb42a14c9e admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:04:37.875521 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 819/1651] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 26 11:04:37 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 11:04:37.880318 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ddf94e-78c6-4694-b9e1-745a48af87c6 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:04:37.897235 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ddf94e-78c6-4694-b9e1-745a48af87c6 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:04:37.910139 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae669917-334c-402e-a547-6588ca8bb161 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:04:37.912480 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae669917-334c-402e-a547-6588ca8bb161 admin admin] REQUEST_METHOD: `PUT` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:04:37.912890 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae669917-334c-402e-a547-6588ca8bb161 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:04:37.914538 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae669917-334c-402e-a547-6588ca8bb161 admin admin] PATH_INFO: `/v3/projects/9d83ece3d0ea4d83a5b0c1949e2c18ff/users/b405953462ea413e8bb3cd1586f96d7e/roles/23e174ce743b46eabe618c212691da55` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:04:37.946905 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae669917-334c-402e-a547-6588ca8bb161 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9d83ece3d0ea4d83a5b0c1949e2c18ff, user_id=b405953462ea413e8bb3cd1586f96d7e, role_id=23e174ce743b46eabe618c212691da55)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:04:37.951613 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae669917-334c-402e-a547-6588ca8bb161 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:04:37.994977 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df65ccb-479c-4e0d-a867-cca84d6e3e88 None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:04:37.995181 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 833/1652] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 26 11:04:37 2022] PUT /identity/v3/projects/9d83ece3d0ea4d83a5b0c1949e2c18ff/users/b405953462ea413e8bb3cd1586f96d7e/roles/23e174ce743b46eabe618c212691da55 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 11:04:37.995732 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df65ccb-479c-4e0d-a867-cca84d6e3e88 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:04:37.996146 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df65ccb-479c-4e0d-a867-cca84d6e3e88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:04:38.070776 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-7df65ccb-479c-4e0d-a867-cca84d6e3e88 None None] MFA Rules not processed for user `b405953462ea413e8bb3cd1586f96d7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 11:04:38.072036 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-7df65ccb-479c-4e0d-a867-cca84d6e3e88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:04:38.309655 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 820/1653] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 11:04:37 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 316 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 11:04:38.315469 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae669917-334c-402e-a547-6588ca8bb161 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:04:38.548859 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae669917-334c-402e-a547-6588ca8bb161 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:04:38.568667 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54355361-51b1-4a03-903d-2c2a94f47361 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:04:38.569560 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54355361-51b1-4a03-903d-2c2a94f47361 admin admin] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:04:38.570774 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54355361-51b1-4a03-903d-2c2a94f47361 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:04:38.572253 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54355361-51b1-4a03-903d-2c2a94f47361 admin admin] PATH_INFO: `/v3/projects` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:04:38.574504 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54355361-51b1-4a03-903d-2c2a94f47361 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:04:38.590542 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54355361-51b1-4a03-903d-2c2a94f47361 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:04:38.675835 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 834/1654] 10.222.0.40 () {60 vars in 1153 bytes} [Fri Aug 26 11:04:38 2022] POST /identity/v3/projects => generated 387 bytes in 362 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 11:04:38.682320 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df65ccb-479c-4e0d-a867-cca84d6e3e88 None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:04:38.707228 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df65ccb-479c-4e0d-a867-cca84d6e3e88 None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:04:38.769189 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 11:04:38.823507 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0df430-b3e9-4400-ac99-f310aaff1f1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:04:38.824250 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0df430-b3e9-4400-ac99-f310aaff1f1c admin admin] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:04:38.824590 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0df430-b3e9-4400-ac99-f310aaff1f1c admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:04:38.825073 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0df430-b3e9-4400-ac99-f310aaff1f1c admin admin] PATH_INFO: `/v3/users` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:04:38.825766 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da0df430-b3e9-4400-ac99-f310aaff1f1c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:04:38.827982 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da0df430-b3e9-4400-ac99-f310aaff1f1c admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:04:38.873566 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54355361-51b1-4a03-903d-2c2a94f47361 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:04:38.874149 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 821/1655] 10.222.0.40 () {60 vars in 1144 bytes} [Fri Aug 26 11:04:38 2022] POST /identity/v3/users => generated 300 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 11:04:38.892661 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54355361-51b1-4a03-903d-2c2a94f47361 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:04:38.900288 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6683cb8d-4617-43be-bb73-5d637e6dc3b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:04:38.900923 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6683cb8d-4617-43be-bb73-5d637e6dc3b2 admin admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:04:38.901206 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6683cb8d-4617-43be-bb73-5d637e6dc3b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:04:38.901474 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6683cb8d-4617-43be-bb73-5d637e6dc3b2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:04:38.902215 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6683cb8d-4617-43be-bb73-5d637e6dc3b2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:04:38.903866 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6683cb8d-4617-43be-bb73-5d637e6dc3b2 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:04:38.939975 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 835/1656] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 26 11:04:38 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 11:04:38.944240 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0df430-b3e9-4400-ac99-f310aaff1f1c admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:04:38.966027 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0df430-b3e9-4400-ac99-f310aaff1f1c admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:04:38.990799 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa51662-788c-49ca-b398-7362ba217a74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:04:38.992660 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa51662-788c-49ca-b398-7362ba217a74 admin admin] REQUEST_METHOD: `PUT` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:04:38.994774 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa51662-788c-49ca-b398-7362ba217a74 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:04:38.995152 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa51662-788c-49ca-b398-7362ba217a74 admin admin] PATH_INFO: `/v3/projects/9db0b7a9abc04534abdcc2e9ca1be8e9/users/64c64fb44f944c2bbab902642becf1ea/roles/1f97e4602e2e4d6498913a81f9ae8d6a` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:04:39.047217 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa51662-788c-49ca-b398-7362ba217a74 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9db0b7a9abc04534abdcc2e9ca1be8e9, user_id=64c64fb44f944c2bbab902642becf1ea, role_id=1f97e4602e2e4d6498913a81f9ae8d6a)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:04:39.056261 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa51662-788c-49ca-b398-7362ba217a74 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:04:39.096312 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 822/1657] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 26 11:04:38 2022] PUT /identity/v3/projects/9db0b7a9abc04534abdcc2e9ca1be8e9/users/64c64fb44f944c2bbab902642becf1ea/roles/1f97e4602e2e4d6498913a81f9ae8d6a => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 11:04:39.097238 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6683cb8d-4617-43be-bb73-5d637e6dc3b2 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:04:39.121353 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6683cb8d-4617-43be-bb73-5d637e6dc3b2 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:04:39.181041 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 11:04:39.280080 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8614f44-ff45-4ec9-85c2-cb3bd59647d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:04:39.282253 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8614f44-ff45-4ec9-85c2-cb3bd59647d2 admin admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:04:39.282253 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8614f44-ff45-4ec9-85c2-cb3bd59647d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:04:39.282253 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8614f44-ff45-4ec9-85c2-cb3bd59647d2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:04:39.282253 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8614f44-ff45-4ec9-85c2-cb3bd59647d2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:04:39.290062 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8614f44-ff45-4ec9-85c2-cb3bd59647d2 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:04:39.317932 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 836/1658] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 26 11:04:39 2022] GET /identity/v3/roles => generated 1420 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 26 11:04:39.323127 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa51662-788c-49ca-b398-7362ba217a74 admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:04:39.343511 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa51662-788c-49ca-b398-7362ba217a74 admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:04:39.374708 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba32b74-8f35-4397-b13e-0d0d91640f64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:04:39.375456 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba32b74-8f35-4397-b13e-0d0d91640f64 admin admin] REQUEST_METHOD: `PUT` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:04:39.375697 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba32b74-8f35-4397-b13e-0d0d91640f64 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:04:39.375983 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba32b74-8f35-4397-b13e-0d0d91640f64 admin admin] PATH_INFO: `/v3/projects/9db0b7a9abc04534abdcc2e9ca1be8e9/users/64c64fb44f944c2bbab902642becf1ea/roles/1f97e4602e2e4d6498913a81f9ae8d6a` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:04:39.389047 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba32b74-8f35-4397-b13e-0d0d91640f64 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9db0b7a9abc04534abdcc2e9ca1be8e9, user_id=64c64fb44f944c2bbab902642becf1ea, role_id=1f97e4602e2e4d6498913a81f9ae8d6a)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:04:39.409481 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba32b74-8f35-4397-b13e-0d0d91640f64 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:04:39.444638 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 823/1659] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 26 11:04:39 2022] PUT /identity/v3/projects/9db0b7a9abc04534abdcc2e9ca1be8e9/users/64c64fb44f944c2bbab902642becf1ea/roles/1f97e4602e2e4d6498913a81f9ae8d6a => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 11:04:39.456397 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34648d52-4598-450a-acc5-7a89374647d9 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:04:39.457063 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34648d52-4598-450a-acc5-7a89374647d9 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:04:39.457063 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34648d52-4598-450a-acc5-7a89374647d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:04:39.533064 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-34648d52-4598-450a-acc5-7a89374647d9 None None] MFA Rules not processed for user `64c64fb44f944c2bbab902642becf1ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 11:04:39.534646 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-34648d52-4598-450a-acc5-7a89374647d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:04:39.749603 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 837/1660] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 11:04:39 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 294 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 11:04:39.764192 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba32b74-8f35-4397-b13e-0d0d91640f64 admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:04:39.766599 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-9ba32b74-8f35-4397-b13e-0d0d91640f64 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:04:39.890926 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba32b74-8f35-4397-b13e-0d0d91640f64 admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:04:39.903526 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dfe11ab-e101-4d31-8410-ae75b295ddec service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c277338a7f814cc6b01c5419775031bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e30ccf0c1204727b786876c141dbcd2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:04:39.904437 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfe11ab-e101-4d31-8410-ae75b295ddec service cinder] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:04:39.904437 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfe11ab-e101-4d31-8410-ae75b295ddec service cinder] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:04:39.905231 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfe11ab-e101-4d31-8410-ae75b295ddec service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:04:39.921082 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dfe11ab-e101-4d31-8410-ae75b295ddec service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:04:39.922712 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dfe11ab-e101-4d31-8410-ae75b295ddec service cinder] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:04:39.950221 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 824/1661] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 11:04:39 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 177 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 11:04:39.979032 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34648d52-4598-450a-acc5-7a89374647d9 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:04:40.099892 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34648d52-4598-450a-acc5-7a89374647d9 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:04:40.105837 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72016c2e-49cf-4edf-accb-4b5f1e82c915 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:04:40.106544 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72016c2e-49cf-4edf-accb-4b5f1e82c915 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:04:40.106820 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72016c2e-49cf-4edf-accb-4b5f1e82c915 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:04:40.107089 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72016c2e-49cf-4edf-accb-4b5f1e82c915 admin admin] PATH_INFO: `/v3/users/b405953462ea413e8bb3cd1586f96d7e` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:04:40.108301 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72016c2e-49cf-4edf-accb-4b5f1e82c915 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b405953462ea413e8bb3cd1586f96d7e)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:04:40.110322 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72016c2e-49cf-4edf-accb-4b5f1e82c915 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:04:40.207900 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 11:04:40.224453 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-72016c2e-49cf-4edf-accb-4b5f1e82c915 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b405953462ea413e8bb3cd1586f96d7e'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:04:40.226227 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-72016c2e-49cf-4edf-accb-4b5f1e82c915 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b405953462ea413e8bb3cd1586f96d7e'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:04:40.239383 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-72016c2e-49cf-4edf-accb-4b5f1e82c915 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b405953462ea413e8bb3cd1586f96d7e'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:04:40.240939 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-72016c2e-49cf-4edf-accb-4b5f1e82c915 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b405953462ea413e8bb3cd1586f96d7e'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:04:40.285841 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-72016c2e-49cf-4edf-accb-4b5f1e82c915 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b405953462ea413e8bb3cd1586f96d7e'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:04:40.326599 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 838/1662] 10.222.0.40 () {58 vars in 1224 bytes} [Fri Aug 26 11:04:39 2022] DELETE /identity/v3/users/b405953462ea413e8bb3cd1586f96d7e => generated 0 bytes in 348 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 11:04:40.330921 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b797d8-9dbe-42f0-81ec-47f6724cddc3 None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:04:40.331146 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b797d8-9dbe-42f0-81ec-47f6724cddc3 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:04:40.331426 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b797d8-9dbe-42f0-81ec-47f6724cddc3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:04:40.367753 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-37b797d8-9dbe-42f0-81ec-47f6724cddc3 None None] MFA Rules not processed for user `ca155a48e2024c32af3fd2e19e5fac11`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 11:04:40.369147 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-37b797d8-9dbe-42f0-81ec-47f6724cddc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:04:40.552894 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 825/1663] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 11:04:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 11:04:40.569163 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72016c2e-49cf-4edf-accb-4b5f1e82c915 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:04:40.571823 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-72016c2e-49cf-4edf-accb-4b5f1e82c915 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:04:40.650420 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72016c2e-49cf-4edf-accb-4b5f1e82c915 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:04:40.655446 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f54f0eb-b475-4db9-9855-210837441c52 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91da2acd2534423091537e1ba068d2b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e30ccf0c1204727b786876c141dbcd2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:04:40.656417 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f54f0eb-b475-4db9-9855-210837441c52 service neutron] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:04:40.657143 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f54f0eb-b475-4db9-9855-210837441c52 service neutron] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:04:40.657645 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f54f0eb-b475-4db9-9855-210837441c52 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:04:40.682383 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f54f0eb-b475-4db9-9855-210837441c52 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:04:40.684740 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f54f0eb-b475-4db9-9855-210837441c52 service neutron] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:04:40.699427 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 839/1664] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 11:04:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 11:04:40.924364 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b797d8-9dbe-42f0-81ec-47f6724cddc3 None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:04:40.926823 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-37b797d8-9dbe-42f0-81ec-47f6724cddc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:04:40.951661 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b797d8-9dbe-42f0-81ec-47f6724cddc3 None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:04:40.962263 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b3a276-e189-4112-9e36-99c0841d7976 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:04:40.963174 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b3a276-e189-4112-9e36-99c0841d7976 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:04:40.963621 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b3a276-e189-4112-9e36-99c0841d7976 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:04:40.964043 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b3a276-e189-4112-9e36-99c0841d7976 admin admin] PATH_INFO: `/v3/projects/a3707cc107d343f9a1fd2df1ddc89441` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:04:40.965548 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b3a276-e189-4112-9e36-99c0841d7976 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a3707cc107d343f9a1fd2df1ddc89441)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:04:40.967999 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b3a276-e189-4112-9e36-99c0841d7976 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:04:41.014266 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 11:04:41.088255 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-82b3a276-e189-4112-9e36-99c0841d7976 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a3707cc107d343f9a1fd2df1ddc89441', 'request_id': 'req-82b3a276-e189-4112-9e36-99c0841d7976'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:04:41.097742 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-82b3a276-e189-4112-9e36-99c0841d7976 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a3707cc107d343f9a1fd2df1ddc89441', 'request_id': 'req-82b3a276-e189-4112-9e36-99c0841d7976'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:04:41.116055 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-82b3a276-e189-4112-9e36-99c0841d7976 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a3707cc107d343f9a1fd2df1ddc89441', 'request_id': 'req-82b3a276-e189-4112-9e36-99c0841d7976'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:04:41.140059 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-82b3a276-e189-4112-9e36-99c0841d7976 admin admin] The token cache is being invalidate because project a3707cc107d343f9a1fd2df1ddc89441 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81423) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 11:04:41.140352 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-82b3a276-e189-4112-9e36-99c0841d7976 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:04:41.141654 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-82b3a276-e189-4112-9e36-99c0841d7976 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:04:41.154541 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 826/1665] 10.222.0.40 () {58 vars in 1233 bytes} [Fri Aug 26 11:04:40 2022] DELETE /identity/v3/projects/a3707cc107d343f9a1fd2df1ddc89441 => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 11:04:41.156325 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43de9576-dabd-4422-830a-36771b0b23ff None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:04:41.156603 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43de9576-dabd-4422-830a-36771b0b23ff None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:04:41.156881 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43de9576-dabd-4422-830a-36771b0b23ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:04:41.192418 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-43de9576-dabd-4422-830a-36771b0b23ff None None] MFA Rules not processed for user `ca155a48e2024c32af3fd2e19e5fac11`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 11:04:41.193749 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-43de9576-dabd-4422-830a-36771b0b23ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:04:41.336055 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 840/1666] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 11:04:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 11:04:41.344479 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b9b2e9-a845-48ad-9779-ac1c5cfa329b None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:04:41.344901 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b9b2e9-a845-48ad-9779-ac1c5cfa329b None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:04:41.345332 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b9b2e9-a845-48ad-9779-ac1c5cfa329b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:04:41.391692 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-95b9b2e9-a845-48ad-9779-ac1c5cfa329b None None] MFA Rules not processed for user `ca155a48e2024c32af3fd2e19e5fac11`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 11:04:41.392952 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-95b9b2e9-a845-48ad-9779-ac1c5cfa329b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:04:41.414366 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 827/1667] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 11:04:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 67 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 11:04:41.414860 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43de9576-dabd-4422-830a-36771b0b23ff None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:04:41.428173 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43de9576-dabd-4422-830a-36771b0b23ff None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:04:41.441788 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c350b3e-e383-4db0-997b-8b8a206fd69f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:04:41.442855 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c350b3e-e383-4db0-997b-8b8a206fd69f admin admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:04:41.442855 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c350b3e-e383-4db0-997b-8b8a206fd69f admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:04:41.443024 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c350b3e-e383-4db0-997b-8b8a206fd69f admin admin] PATH_INFO: `/v3/domains` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:04:41.443748 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c350b3e-e383-4db0-997b-8b8a206fd69f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 11:04:41.444248 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c350b3e-e383-4db0-997b-8b8a206fd69f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:04:41.449290 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c350b3e-e383-4db0-997b-8b8a206fd69f admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:04:41.481509 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 841/1668] 10.222.0.40 () {58 vars in 1153 bytes} [Fri Aug 26 11:04:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 11:04:41.482710 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b9b2e9-a845-48ad-9779-ac1c5cfa329b None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:04:41.493264 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b9b2e9-a845-48ad-9779-ac1c5cfa329b None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:04:41.506731 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746878b2-217d-4401-b535-a419a01d62c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:04:41.508087 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746878b2-217d-4401-b535-a419a01d62c6 admin admin] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:04:41.508388 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746878b2-217d-4401-b535-a419a01d62c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:04:41.508643 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746878b2-217d-4401-b535-a419a01d62c6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:04:41.509291 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-746878b2-217d-4401-b535-a419a01d62c6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:04:41.511288 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-746878b2-217d-4401-b535-a419a01d62c6 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:04:41.577018 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 828/1669] 10.222.0.40 () {60 vars in 1153 bytes} [Fri Aug 26 11:04:41 2022] POST /identity/v3/projects => generated 373 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 11:04:41.581547 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c350b3e-e383-4db0-997b-8b8a206fd69f admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:04:41.598218 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c350b3e-e383-4db0-997b-8b8a206fd69f admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:04:41.727668 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=96,keystone:DELETE=3,keystone:INSERT=1 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 11:04:41.748732 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72df6d8-485c-4287-8441-e4f1f84fa4ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:04:41.749550 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72df6d8-485c-4287-8441-e4f1f84fa4ea admin admin] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:04:41.749991 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72df6d8-485c-4287-8441-e4f1f84fa4ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:04:41.750284 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72df6d8-485c-4287-8441-e4f1f84fa4ea admin admin] PATH_INFO: `/v3/users` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:04:41.755314 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b72df6d8-485c-4287-8441-e4f1f84fa4ea admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:04:41.757854 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b72df6d8-485c-4287-8441-e4f1f84fa4ea admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:04:41.815579 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 842/1670] 10.222.0.40 () {60 vars in 1144 bytes} [Fri Aug 26 11:04:41 2022] POST /identity/v3/users => generated 287 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 11:04:41.820072 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746878b2-217d-4401-b535-a419a01d62c6 admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:04:41.836243 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746878b2-217d-4401-b535-a419a01d62c6 admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:04:41.845696 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ff6f8f-eb2d-4505-b948-089912b9a05a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:04:41.847186 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ff6f8f-eb2d-4505-b948-089912b9a05a admin admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:04:41.848404 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ff6f8f-eb2d-4505-b948-089912b9a05a admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:04:41.851415 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ff6f8f-eb2d-4505-b948-089912b9a05a admin admin] PATH_INFO: `/v3/roles` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:04:41.852393 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ff6f8f-eb2d-4505-b948-089912b9a05a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:04:41.863228 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ff6f8f-eb2d-4505-b948-089912b9a05a admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:04:41.902157 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 829/1671] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 26 11:04:41 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 11:04:41.907290 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72df6d8-485c-4287-8441-e4f1f84fa4ea admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:04:41.922495 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72df6d8-485c-4287-8441-e4f1f84fa4ea admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:04:41.934754 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d95d62d-9609-4a68-8056-860571771cdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:04:41.935592 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d95d62d-9609-4a68-8056-860571771cdc admin admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:04:41.936021 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d95d62d-9609-4a68-8056-860571771cdc admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:04:41.936432 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d95d62d-9609-4a68-8056-860571771cdc admin admin] PATH_INFO: `/v3/roles` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:04:41.937222 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d95d62d-9609-4a68-8056-860571771cdc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:04:41.939176 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d95d62d-9609-4a68-8056-860571771cdc admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:04:41.964931 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 843/1672] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 26 11:04:41 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 11:04:41.969415 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ff6f8f-eb2d-4505-b948-089912b9a05a admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:04:41.983921 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ff6f8f-eb2d-4505-b948-089912b9a05a admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:04:41.998049 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6717995a-5203-4356-b23d-84369eeccd96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:04:41.998628 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6717995a-5203-4356-b23d-84369eeccd96 admin admin] REQUEST_METHOD: `PUT` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:04:41.998914 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6717995a-5203-4356-b23d-84369eeccd96 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:04:41.999805 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6717995a-5203-4356-b23d-84369eeccd96 admin admin] PATH_INFO: `/v3/projects/16700221d9794d5bab7029250b3e39ee/users/b6ae4c4e15944c768c6cca86f057cf22/roles/23e174ce743b46eabe618c212691da55` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:04:42.023720 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6717995a-5203-4356-b23d-84369eeccd96 admin admin] RBAC: Authorizing `identity:create_grant(project_id=16700221d9794d5bab7029250b3e39ee, user_id=b6ae4c4e15944c768c6cca86f057cf22, role_id=23e174ce743b46eabe618c212691da55)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:04:42.027087 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6717995a-5203-4356-b23d-84369eeccd96 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:04:42.048207 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 830/1673] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 26 11:04:41 2022] PUT /identity/v3/projects/16700221d9794d5bab7029250b3e39ee/users/b6ae4c4e15944c768c6cca86f057cf22/roles/23e174ce743b46eabe618c212691da55 => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 11:04:42.059614 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef1fbc8-69ab-466e-8d68-a3e2ae194872 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:04:42.059889 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef1fbc8-69ab-466e-8d68-a3e2ae194872 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:04:42.060159 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef1fbc8-69ab-466e-8d68-a3e2ae194872 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:04:42.119939 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-cef1fbc8-69ab-466e-8d68-a3e2ae194872 None None] MFA Rules not processed for user `b6ae4c4e15944c768c6cca86f057cf22`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 11:04:42.121196 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-cef1fbc8-69ab-466e-8d68-a3e2ae194872 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:04:42.258657 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 844/1674] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 11:04:42 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 11:04:42.264126 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6717995a-5203-4356-b23d-84369eeccd96 admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:04:42.389652 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef1fbc8-69ab-466e-8d68-a3e2ae194872 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:04:42.391505 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-cef1fbc8-69ab-466e-8d68-a3e2ae194872 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:04:42.460835 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6717995a-5203-4356-b23d-84369eeccd96 admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:04:42.475290 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b0ac50-bb9a-4004-930f-d85fc79f48a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:04:42.476535 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b0ac50-bb9a-4004-930f-d85fc79f48a5 admin admin] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:04:42.476914 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b0ac50-bb9a-4004-930f-d85fc79f48a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:04:42.477185 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b0ac50-bb9a-4004-930f-d85fc79f48a5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:04:42.477847 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b0ac50-bb9a-4004-930f-d85fc79f48a5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:04:42.479792 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b0ac50-bb9a-4004-930f-d85fc79f48a5 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:04:42.534504 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 831/1675] 10.222.0.40 () {60 vars in 1153 bytes} [Fri Aug 26 11:04:42 2022] POST /identity/v3/projects => generated 371 bytes in 269 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 26 11:04:42.535323 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b0ac50-bb9a-4004-930f-d85fc79f48a5 admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:04:42.590509 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef1fbc8-69ab-466e-8d68-a3e2ae194872 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:04:42.595256 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 11:04:42.683816 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b0ac50-bb9a-4004-930f-d85fc79f48a5 admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:04:42.706361 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d16b037-2d0c-4267-8793-50dff1c93314 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:04:42.706930 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d16b037-2d0c-4267-8793-50dff1c93314 admin admin] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:04:42.707214 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d16b037-2d0c-4267-8793-50dff1c93314 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:04:42.707474 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d16b037-2d0c-4267-8793-50dff1c93314 admin admin] PATH_INFO: `/v3/users` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:04:42.709122 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d16b037-2d0c-4267-8793-50dff1c93314 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:04:42.711811 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d16b037-2d0c-4267-8793-50dff1c93314 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:04:42.800766 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 832/1676] 10.222.0.40 () {60 vars in 1144 bytes} [Fri Aug 26 11:04:42 2022] POST /identity/v3/users => generated 292 bytes in 265 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 11:04:42.801780 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5532473c-5b59-47df-974f-05e832dee54c None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:04:42.802084 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5532473c-5b59-47df-974f-05e832dee54c None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:04:42.802689 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5532473c-5b59-47df-974f-05e832dee54c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:04:42.838398 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112120e8-4261-4f3a-b194-30b23609d552 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:04:42.840308 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112120e8-4261-4f3a-b194-30b23609d552 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:04:42.841162 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112120e8-4261-4f3a-b194-30b23609d552 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:04:42.841675 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112120e8-4261-4f3a-b194-30b23609d552 admin admin] PATH_INFO: `/v3/projects/9d83ece3d0ea4d83a5b0c1949e2c18ff` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:04:42.845224 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-112120e8-4261-4f3a-b194-30b23609d552 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9d83ece3d0ea4d83a5b0c1949e2c18ff)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:04:42.855073 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-112120e8-4261-4f3a-b194-30b23609d552 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:04:42.886066 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 11:04:42.895909 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-5532473c-5b59-47df-974f-05e832dee54c None None] MFA Rules not processed for user `d6dbbabdd88c4c088bc91e6c43531318`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 11:04:42.902154 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-5532473c-5b59-47df-974f-05e832dee54c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:04:43.003274 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-112120e8-4261-4f3a-b194-30b23609d552 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9d83ece3d0ea4d83a5b0c1949e2c18ff', 'request_id': 'req-112120e8-4261-4f3a-b194-30b23609d552'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:04:43.021305 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-112120e8-4261-4f3a-b194-30b23609d552 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9d83ece3d0ea4d83a5b0c1949e2c18ff', 'request_id': 'req-112120e8-4261-4f3a-b194-30b23609d552'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:04:43.038916 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-112120e8-4261-4f3a-b194-30b23609d552 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9d83ece3d0ea4d83a5b0c1949e2c18ff', 'request_id': 'req-112120e8-4261-4f3a-b194-30b23609d552'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:04:43.070964 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-112120e8-4261-4f3a-b194-30b23609d552 admin admin] The token cache is being invalidate because project 9d83ece3d0ea4d83a5b0c1949e2c18ff was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81424) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 11:04:43.071312 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-112120e8-4261-4f3a-b194-30b23609d552 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:04:43.071679 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 833/1677] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 11:04:42 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 271 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 11:04:43.073247 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5532473c-5b59-47df-974f-05e832dee54c None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:04:43.082292 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-112120e8-4261-4f3a-b194-30b23609d552 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:04:43.089136 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 845/1678] 10.222.0.40 () {58 vars in 1233 bytes} [Fri Aug 26 11:04:42 2022] DELETE /identity/v3/projects/9d83ece3d0ea4d83a5b0c1949e2c18ff => generated 0 bytes in 698 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 11:04:43.096133 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a857ee6-3466-4165-be5c-8c08ba9394c7 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:04:43.096424 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a857ee6-3466-4165-be5c-8c08ba9394c7 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:04:43.097401 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a857ee6-3466-4165-be5c-8c08ba9394c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:04:43.113526 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5532473c-5b59-47df-974f-05e832dee54c None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:04:43.141066 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-7a857ee6-3466-4165-be5c-8c08ba9394c7 None None] MFA Rules not processed for user `d6dbbabdd88c4c088bc91e6c43531318`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 11:04:43.142442 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-7a857ee6-3466-4165-be5c-8c08ba9394c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:04:43.228180 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee572f65-1c01-4de3-b95f-1ac29a82d783 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:04:43.229082 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee572f65-1c01-4de3-b95f-1ac29a82d783 admin admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:04:43.229553 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee572f65-1c01-4de3-b95f-1ac29a82d783 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:04:43.230010 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee572f65-1c01-4de3-b95f-1ac29a82d783 admin admin] PATH_INFO: `/v3/roles` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:04:43.230840 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee572f65-1c01-4de3-b95f-1ac29a82d783 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:04:43.232829 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee572f65-1c01-4de3-b95f-1ac29a82d783 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:04:43.244262 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 834/1679] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 26 11:04:43 2022] GET /identity/v3/roles => generated 1420 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 11:04:43.245429 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee572f65-1c01-4de3-b95f-1ac29a82d783 admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:04:43.251864 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-ee572f65-1c01-4de3-b95f-1ac29a82d783 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:04:43.267119 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee572f65-1c01-4de3-b95f-1ac29a82d783 admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:04:43.275092 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df09e1ff-1e9b-41a6-a57d-e7c413bbfe49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:04:43.275879 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df09e1ff-1e9b-41a6-a57d-e7c413bbfe49 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:04:43.276224 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df09e1ff-1e9b-41a6-a57d-e7c413bbfe49 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:04:43.276661 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df09e1ff-1e9b-41a6-a57d-e7c413bbfe49 admin admin] PATH_INFO: `/v3/users/64c64fb44f944c2bbab902642becf1ea` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:04:43.277926 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df09e1ff-1e9b-41a6-a57d-e7c413bbfe49 admin admin] RBAC: Authorizing `identity:delete_user(user_id=64c64fb44f944c2bbab902642becf1ea)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:04:43.282255 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df09e1ff-1e9b-41a6-a57d-e7c413bbfe49 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:04:43.294015 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 846/1680] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 11:04:43 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 11:04:43.295411 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a857ee6-3466-4165-be5c-8c08ba9394c7 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:04:43.300086 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-7a857ee6-3466-4165-be5c-8c08ba9394c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:04:43.327585 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a857ee6-3466-4165-be5c-8c08ba9394c7 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:04:43.343155 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4022b681-2861-4f32-a673-75473398acfc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:04:43.343862 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4022b681-2861-4f32-a673-75473398acfc admin admin] REQUEST_METHOD: `PUT` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:04:43.344157 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4022b681-2861-4f32-a673-75473398acfc admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:04:43.344432 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4022b681-2861-4f32-a673-75473398acfc admin admin] PATH_INFO: `/v3/projects/dd36393983104344bda41470fe7358f3/users/65fd1dae078f41138e6c5a958e0972fc/roles/1f97e4602e2e4d6498913a81f9ae8d6a` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:04:43.344655 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=92,keystone:INSERT=3,keystone:DELETE=5 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 11:04:43.380677 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-df09e1ff-1e9b-41a6-a57d-e7c413bbfe49 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '64c64fb44f944c2bbab902642becf1ea'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:04:43.404236 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-df09e1ff-1e9b-41a6-a57d-e7c413bbfe49 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '64c64fb44f944c2bbab902642becf1ea'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:04:43.405137 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-df09e1ff-1e9b-41a6-a57d-e7c413bbfe49 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '64c64fb44f944c2bbab902642becf1ea'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:04:43.412360 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4022b681-2861-4f32-a673-75473398acfc admin admin] RBAC: Authorizing `identity:create_grant(project_id=dd36393983104344bda41470fe7358f3, user_id=65fd1dae078f41138e6c5a958e0972fc, role_id=1f97e4602e2e4d6498913a81f9ae8d6a)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:04:43.416931 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4022b681-2861-4f32-a673-75473398acfc admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:04:43.453283 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 847/1681] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 26 11:04:43 2022] PUT /identity/v3/projects/dd36393983104344bda41470fe7358f3/users/65fd1dae078f41138e6c5a958e0972fc/roles/1f97e4602e2e4d6498913a81f9ae8d6a => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 11:04:43.453990 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-df09e1ff-1e9b-41a6-a57d-e7c413bbfe49 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '64c64fb44f944c2bbab902642becf1ea'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:04:43.454675 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-df09e1ff-1e9b-41a6-a57d-e7c413bbfe49 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '64c64fb44f944c2bbab902642becf1ea'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:04:43.455161 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4022b681-2861-4f32-a673-75473398acfc admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:04:43.458041 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-4022b681-2861-4f32-a673-75473398acfc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:04:43.499628 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 835/1682] 10.222.0.40 () {58 vars in 1224 bytes} [Fri Aug 26 11:04:43 2022] DELETE /identity/v3/users/64c64fb44f944c2bbab902642becf1ea => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 11:04:43.500846 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df09e1ff-1e9b-41a6-a57d-e7c413bbfe49 admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:04:43.506682 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-df09e1ff-1e9b-41a6-a57d-e7c413bbfe49 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:04:43.582726 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4022b681-2861-4f32-a673-75473398acfc admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:04:43.605361 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 11:04:43.670161 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df09e1ff-1e9b-41a6-a57d-e7c413bbfe49 admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:04:43.677477 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a513dc6e-216f-4729-90fc-c0a32e5c8c41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:04:43.679527 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a513dc6e-216f-4729-90fc-c0a32e5c8c41 admin admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:04:43.680020 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a513dc6e-216f-4729-90fc-c0a32e5c8c41 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:04:43.680492 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a513dc6e-216f-4729-90fc-c0a32e5c8c41 admin admin] PATH_INFO: `/v3/roles` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:04:43.682909 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a513dc6e-216f-4729-90fc-c0a32e5c8c41 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:04:43.686902 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a513dc6e-216f-4729-90fc-c0a32e5c8c41 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:04:43.716612 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 836/1683] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 26 11:04:43 2022] GET /identity/v3/roles => generated 1420 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 11:04:43.719399 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a513dc6e-216f-4729-90fc-c0a32e5c8c41 admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:04:43.735202 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912b3521-ebd7-443e-955c-697cd1242348 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f4b4a662db049ecacc865ecff10bdb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e30ccf0c1204727b786876c141dbcd2', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:04:43.737387 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912b3521-ebd7-443e-955c-697cd1242348 service nova] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:04:43.737931 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912b3521-ebd7-443e-955c-697cd1242348 service nova] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:04:43.739543 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912b3521-ebd7-443e-955c-697cd1242348 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:04:43.743043 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-912b3521-ebd7-443e-955c-697cd1242348 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:04:43.746450 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a513dc6e-216f-4729-90fc-c0a32e5c8c41 admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:04:43.767587 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b0613b-bf1b-4925-ae43-aae7dde177c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:04:43.768283 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b0613b-bf1b-4925-ae43-aae7dde177c8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:04:43.768787 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b0613b-bf1b-4925-ae43-aae7dde177c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:04:43.778555 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b0613b-bf1b-4925-ae43-aae7dde177c8 admin admin] PATH_INFO: `/v3/projects/dd36393983104344bda41470fe7358f3/users/65fd1dae078f41138e6c5a958e0972fc/roles/1f97e4602e2e4d6498913a81f9ae8d6a` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:04:43.784240 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b0613b-bf1b-4925-ae43-aae7dde177c8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dd36393983104344bda41470fe7358f3, user_id=65fd1dae078f41138e6c5a958e0972fc, role_id=1f97e4602e2e4d6498913a81f9ae8d6a)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:04:43.796755 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b0613b-bf1b-4925-ae43-aae7dde177c8 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:04:43.845830 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 837/1684] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 26 11:04:43 2022] PUT /identity/v3/projects/dd36393983104344bda41470fe7358f3/users/65fd1dae078f41138e6c5a958e0972fc/roles/1f97e4602e2e4d6498913a81f9ae8d6a => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 11:04:43.847633 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ad55b4-6c55-4b7f-8332-f4bbb34dbf1c None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:04:43.848032 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ad55b4-6c55-4b7f-8332-f4bbb34dbf1c None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:04:43.848409 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ad55b4-6c55-4b7f-8332-f4bbb34dbf1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:04:43.958231 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-02ad55b4-6c55-4b7f-8332-f4bbb34dbf1c None None] MFA Rules not processed for user `65fd1dae078f41138e6c5a958e0972fc`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 11:04:43.959729 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-02ad55b4-6c55-4b7f-8332-f4bbb34dbf1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:04:43.974988 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-912b3521-ebd7-443e-955c-697cd1242348 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:04:43.979819 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-912b3521-ebd7-443e-955c-697cd1242348 service nova] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:04:44.024972 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 848/1685] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 11:04:43 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 570 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 11:04:44.184607 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 838/1686] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 11:04:43 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 338 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 11:04:44.199965 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912b3521-ebd7-443e-955c-697cd1242348 service nova] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:04:44.202010 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-912b3521-ebd7-443e-955c-697cd1242348 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:04:44.320982 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912b3521-ebd7-443e-955c-697cd1242348 service nova] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:04:44.452185 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7afc6893-50cf-4724-856c-b9ce2162e772 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c277338a7f814cc6b01c5419775031bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e30ccf0c1204727b786876c141dbcd2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:04:44.454549 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afc6893-50cf-4724-856c-b9ce2162e772 service cinder] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:04:44.454930 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afc6893-50cf-4724-856c-b9ce2162e772 service cinder] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:04:44.455420 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afc6893-50cf-4724-856c-b9ce2162e772 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:04:44.468329 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7afc6893-50cf-4724-856c-b9ce2162e772 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:04:44.469933 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7afc6893-50cf-4724-856c-b9ce2162e772 service cinder] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:04:44.520052 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 849/1687] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 11:04:44 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 11:04:45.446266 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc310aa8-347e-42e3-bf56-0034d44d05f9 None None] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:04:45.446266 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc310aa8-347e-42e3-bf56-0034d44d05f9 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:04:45.446266 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc310aa8-347e-42e3-bf56-0034d44d05f9 None None] PATH_INFO: `/` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:04:45.454208 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 839/1688] 10.222.0.40 () {54 vars in 912 bytes} [Fri Aug 26 11:04:45 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 11:04:45.459478 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0bacd9-39dd-4022-a2ac-d6aa3ccfd99b None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:04:45.460629 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0bacd9-39dd-4022-a2ac-d6aa3ccfd99b None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:04:45.461191 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0bacd9-39dd-4022-a2ac-d6aa3ccfd99b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:04:45.531526 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 11:04:45.571985 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-0e0bacd9-39dd-4022-a2ac-d6aa3ccfd99b None None] MFA Rules not processed for user `91da2acd2534423091537e1ba068d2b7`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 11:04:45.573750 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-0e0bacd9-39dd-4022-a2ac-d6aa3ccfd99b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:04:45.592687 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc310aa8-347e-42e3-bf56-0034d44d05f9 None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:04:45.594511 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-dc310aa8-347e-42e3-bf56-0034d44d05f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:04:45.674306 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=93,keystone:DELETE=5,keystone:INSERT=2 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 11:04:45.723661 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 850/1689] 10.222.0.40 () {58 vars in 1009 bytes} [Fri Aug 26 11:04:45 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 265 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 11:04:45.724760 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0bacd9-39dd-4022-a2ac-d6aa3ccfd99b None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:04:45.844516 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc310aa8-347e-42e3-bf56-0034d44d05f9 None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:04:45.855084 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8024955f-cf04-4673-9fe1-f2d33948cdb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:04:45.857722 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8024955f-cf04-4673-9fe1-f2d33948cdb9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:04:45.858251 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8024955f-cf04-4673-9fe1-f2d33948cdb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:04:45.860207 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8024955f-cf04-4673-9fe1-f2d33948cdb9 admin admin] PATH_INFO: `/v3/users/bbeef9cfedfa4bd386617b92ba1dfc90` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:04:45.862867 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8024955f-cf04-4673-9fe1-f2d33948cdb9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bbeef9cfedfa4bd386617b92ba1dfc90)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:04:45.866092 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8024955f-cf04-4673-9fe1-f2d33948cdb9 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:04:45.904679 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0bacd9-39dd-4022-a2ac-d6aa3ccfd99b None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:04:45.935166 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78928cc-106e-450c-a5e1-f3299340f210 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:04:45.937631 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78928cc-106e-450c-a5e1-f3299340f210 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:04:45.939475 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78928cc-106e-450c-a5e1-f3299340f210 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:04:45.940303 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78928cc-106e-450c-a5e1-f3299340f210 admin admin] PATH_INFO: `/v3/users/b6ae4c4e15944c768c6cca86f057cf22` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:04:45.942633 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d78928cc-106e-450c-a5e1-f3299340f210 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b6ae4c4e15944c768c6cca86f057cf22)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:04:45.946045 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d78928cc-106e-450c-a5e1-f3299340f210 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:04:46.093618 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-8024955f-cf04-4673-9fe1-f2d33948cdb9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bbeef9cfedfa4bd386617b92ba1dfc90'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:04:46.106962 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-d78928cc-106e-450c-a5e1-f3299340f210 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b6ae4c4e15944c768c6cca86f057cf22'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:04:46.108630 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-d78928cc-106e-450c-a5e1-f3299340f210 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b6ae4c4e15944c768c6cca86f057cf22'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:04:46.111546 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-8024955f-cf04-4673-9fe1-f2d33948cdb9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bbeef9cfedfa4bd386617b92ba1dfc90'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:04:46.113251 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-8024955f-cf04-4673-9fe1-f2d33948cdb9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bbeef9cfedfa4bd386617b92ba1dfc90'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:04:46.127511 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-d78928cc-106e-450c-a5e1-f3299340f210 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b6ae4c4e15944c768c6cca86f057cf22'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:04:46.130132 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-d78928cc-106e-450c-a5e1-f3299340f210 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b6ae4c4e15944c768c6cca86f057cf22'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:04:46.150773 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-8024955f-cf04-4673-9fe1-f2d33948cdb9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bbeef9cfedfa4bd386617b92ba1dfc90'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:04:46.152363 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-8024955f-cf04-4673-9fe1-f2d33948cdb9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bbeef9cfedfa4bd386617b92ba1dfc90'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:04:46.172235 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-d78928cc-106e-450c-a5e1-f3299340f210 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b6ae4c4e15944c768c6cca86f057cf22'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:04:46.185467 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 840/1690] 10.222.0.40 () {58 vars in 1224 bytes} [Fri Aug 26 11:04:45 2022] DELETE /identity/v3/users/bbeef9cfedfa4bd386617b92ba1dfc90 => generated 0 bytes in 594 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 11:04:46.186100 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8024955f-cf04-4673-9fe1-f2d33948cdb9 admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:04:46.196749 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-8024955f-cf04-4673-9fe1-f2d33948cdb9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:04:46.216810 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 851/1691] 10.222.0.40 () {58 vars in 1224 bytes} [Fri Aug 26 11:04:45 2022] DELETE /identity/v3/users/b6ae4c4e15944c768c6cca86f057cf22 => generated 0 bytes in 493 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 11:04:46.223476 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9b3c4c-799f-4578-8b94-04b164b3b775 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:04:46.223763 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9b3c4c-799f-4578-8b94-04b164b3b775 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:04:46.224038 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9b3c4c-799f-4578-8b94-04b164b3b775 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:04:46.297483 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-af9b3c4c-799f-4578-8b94-04b164b3b775 None None] MFA Rules not processed for user `d074367ea693406badce6f7b468f3147`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 11:04:46.298569 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-af9b3c4c-799f-4578-8b94-04b164b3b775 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:04:46.335854 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 11:04:46.382495 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8024955f-cf04-4673-9fe1-f2d33948cdb9 admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:04:46.491779 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 852/1692] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 11:04:46 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 274 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 11:04:46.493565 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9b3c4c-799f-4578-8b94-04b164b3b775 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:04:46.495189 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-af9b3c4c-799f-4578-8b94-04b164b3b775 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:04:46.537044 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7671e1-5787-45b9-a6e2-69dfabec9ac3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91da2acd2534423091537e1ba068d2b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e30ccf0c1204727b786876c141dbcd2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:04:46.537671 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7671e1-5787-45b9-a6e2-69dfabec9ac3 service neutron] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:04:46.538991 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7671e1-5787-45b9-a6e2-69dfabec9ac3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:04:46.539770 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7671e1-5787-45b9-a6e2-69dfabec9ac3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:04:46.551717 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-8e7671e1-5787-45b9-a6e2-69dfabec9ac3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:04:46.578972 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9b3c4c-799f-4578-8b94-04b164b3b775 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:04:46.584674 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c02bc7c-6e8d-4ae0-9dd3-2ec21bfe7fe2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:04:46.586206 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c02bc7c-6e8d-4ae0-9dd3-2ec21bfe7fe2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:04:46.586658 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c02bc7c-6e8d-4ae0-9dd3-2ec21bfe7fe2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:04:46.587096 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c02bc7c-6e8d-4ae0-9dd3-2ec21bfe7fe2 admin admin] PATH_INFO: `/v3/projects/9db0b7a9abc04534abdcc2e9ca1be8e9` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:04:46.589354 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c02bc7c-6e8d-4ae0-9dd3-2ec21bfe7fe2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9db0b7a9abc04534abdcc2e9ca1be8e9)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:04:46.591511 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c02bc7c-6e8d-4ae0-9dd3-2ec21bfe7fe2 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:04:46.595206 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e7671e1-5787-45b9-a6e2-69dfabec9ac3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:04:46.596660 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e7671e1-5787-45b9-a6e2-69dfabec9ac3 service neutron] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:04:46.611636 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 11:04:46.622137 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 841/1693] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 11:04:46 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 436 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 11:04:46.623600 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2491bc63-6cde-47bd-9fce-5c2e7132fc6b None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:04:46.623984 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2491bc63-6cde-47bd-9fce-5c2e7132fc6b None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:04:46.624146 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2491bc63-6cde-47bd-9fce-5c2e7132fc6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:04:46.676470 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-2491bc63-6cde-47bd-9fce-5c2e7132fc6b None None] MFA Rules not processed for user `ca155a48e2024c32af3fd2e19e5fac11`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 11:04:46.681936 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-6c02bc7c-6e8d-4ae0-9dd3-2ec21bfe7fe2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9db0b7a9abc04534abdcc2e9ca1be8e9', 'request_id': 'req-6c02bc7c-6e8d-4ae0-9dd3-2ec21bfe7fe2'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:04:46.684092 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-2491bc63-6cde-47bd-9fce-5c2e7132fc6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:04:46.704022 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-6c02bc7c-6e8d-4ae0-9dd3-2ec21bfe7fe2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9db0b7a9abc04534abdcc2e9ca1be8e9', 'request_id': 'req-6c02bc7c-6e8d-4ae0-9dd3-2ec21bfe7fe2'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:04:46.732072 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-6c02bc7c-6e8d-4ae0-9dd3-2ec21bfe7fe2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9db0b7a9abc04534abdcc2e9ca1be8e9', 'request_id': 'req-6c02bc7c-6e8d-4ae0-9dd3-2ec21bfe7fe2'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:04:46.765622 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-6c02bc7c-6e8d-4ae0-9dd3-2ec21bfe7fe2 admin admin] The token cache is being invalidate because project 9db0b7a9abc04534abdcc2e9ca1be8e9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81424) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 11:04:46.765921 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-6c02bc7c-6e8d-4ae0-9dd3-2ec21bfe7fe2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:04:46.766669 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-6c02bc7c-6e8d-4ae0-9dd3-2ec21bfe7fe2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:04:46.774311 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 853/1694] 10.222.0.40 () {58 vars in 1233 bytes} [Fri Aug 26 11:04:46 2022] DELETE /identity/v3/projects/9db0b7a9abc04534abdcc2e9ca1be8e9 => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 11:04:46.775906 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a2fc80-f610-4f56-91a7-5ee0f71ad341 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:04:46.776179 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a2fc80-f610-4f56-91a7-5ee0f71ad341 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:04:46.776460 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a2fc80-f610-4f56-91a7-5ee0f71ad341 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:04:46.846461 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-f6a2fc80-f610-4f56-91a7-5ee0f71ad341 None None] MFA Rules not processed for user `ca155a48e2024c32af3fd2e19e5fac11`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 11:04:46.851565 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-f6a2fc80-f610-4f56-91a7-5ee0f71ad341 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:04:46.863793 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 842/1695] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 11:04:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 241 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 11:04:46.865297 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab0a8f5-6067-4b87-a77f-0e78ec55132c None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:04:46.865812 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab0a8f5-6067-4b87-a77f-0e78ec55132c None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:04:46.867319 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab0a8f5-6067-4b87-a77f-0e78ec55132c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:04:46.892654 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 854/1696] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 11:04:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 11:04:46.899712 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6b70f5-5ed2-46a5-80ed-972324b7a32e None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:04:46.900014 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6b70f5-5ed2-46a5-80ed-972324b7a32e None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:04:46.900287 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6b70f5-5ed2-46a5-80ed-972324b7a32e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:04:46.909021 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-6ab0a8f5-6067-4b87-a77f-0e78ec55132c None None] MFA Rules not processed for user `d074367ea693406badce6f7b468f3147`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 11:04:46.911328 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-6ab0a8f5-6067-4b87-a77f-0e78ec55132c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:04:46.941210 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-cd6b70f5-5ed2-46a5-80ed-972324b7a32e None None] MFA Rules not processed for user `ca155a48e2024c32af3fd2e19e5fac11`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 11:04:46.942642 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-cd6b70f5-5ed2-46a5-80ed-972324b7a32e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:04:46.968231 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 855/1697] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 11:04:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 69 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 11:04:46.968231 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6b70f5-5ed2-46a5-80ed-972324b7a32e None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:04:46.968231 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-cd6b70f5-5ed2-46a5-80ed-972324b7a32e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:04:47.038592 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 843/1698] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 11:04:46 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 11:04:47.039406 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab0a8f5-6067-4b87-a77f-0e78ec55132c None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:04:47.045398 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-6ab0a8f5-6067-4b87-a77f-0e78ec55132c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:04:47.106000 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 11:04:47.144937 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6b70f5-5ed2-46a5-80ed-972324b7a32e None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:04:47.154487 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac77e53-efae-41f5-b84d-f227e68ada34 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91da2acd2534423091537e1ba068d2b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e30ccf0c1204727b786876c141dbcd2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:04:47.155324 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac77e53-efae-41f5-b84d-f227e68ada34 service neutron] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:04:47.155639 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac77e53-efae-41f5-b84d-f227e68ada34 service neutron] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:04:47.156049 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac77e53-efae-41f5-b84d-f227e68ada34 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:04:47.180435 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac77e53-efae-41f5-b84d-f227e68ada34 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:04:47.182675 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac77e53-efae-41f5-b84d-f227e68ada34 service neutron] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:04:47.199360 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 856/1699] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 11:04:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 236 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 11:04:47.201007 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db157c0-8290-46ee-b309-231aa216bc47 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:04:47.201299 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db157c0-8290-46ee-b309-231aa216bc47 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:04:47.201681 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db157c0-8290-46ee-b309-231aa216bc47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:04:47.218923 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab0a8f5-6067-4b87-a77f-0e78ec55132c None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:04:47.242759 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3cf5dd-bad2-4f62-9b20-4f2f256bab0f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91da2acd2534423091537e1ba068d2b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e30ccf0c1204727b786876c141dbcd2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:04:47.243396 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3cf5dd-bad2-4f62-9b20-4f2f256bab0f service neutron] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:04:47.243672 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3cf5dd-bad2-4f62-9b20-4f2f256bab0f service neutron] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:04:47.243945 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3cf5dd-bad2-4f62-9b20-4f2f256bab0f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:04:47.245306 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-1db157c0-8290-46ee-b309-231aa216bc47 None None] MFA Rules not processed for user `ca155a48e2024c32af3fd2e19e5fac11`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 11:04:47.246978 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-1db157c0-8290-46ee-b309-231aa216bc47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:04:47.261497 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 857/1700] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 11:04:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 61 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 11:04:47.263958 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db157c0-8290-46ee-b309-231aa216bc47 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:04:47.265517 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-1db157c0-8290-46ee-b309-231aa216bc47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:04:47.271082 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a3cf5dd-bad2-4f62-9b20-4f2f256bab0f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:04:47.275027 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a3cf5dd-bad2-4f62-9b20-4f2f256bab0f service neutron] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:04:47.314494 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 844/1701] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 11:04:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 275 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 11:04:47.316139 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3cf5dd-bad2-4f62-9b20-4f2f256bab0f service neutron] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:04:47.351464 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3cf5dd-bad2-4f62-9b20-4f2f256bab0f service neutron] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:04:47.371969 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19783bcb-6843-4f3f-8a08-3c258cc4c771 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:04:47.373749 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19783bcb-6843-4f3f-8a08-3c258cc4c771 admin admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:04:47.374631 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19783bcb-6843-4f3f-8a08-3c258cc4c771 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:04:47.374912 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19783bcb-6843-4f3f-8a08-3c258cc4c771 admin admin] PATH_INFO: `/v3/domains` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:04:47.377679 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19783bcb-6843-4f3f-8a08-3c258cc4c771 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 11:04:47.378016 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19783bcb-6843-4f3f-8a08-3c258cc4c771 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:04:47.382321 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19783bcb-6843-4f3f-8a08-3c258cc4c771 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:04:47.415442 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 845/1702] 10.222.0.40 () {58 vars in 1153 bytes} [Fri Aug 26 11:04:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 11:04:47.421810 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19783bcb-6843-4f3f-8a08-3c258cc4c771 admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:04:47.445645 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19783bcb-6843-4f3f-8a08-3c258cc4c771 admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:04:47.458599 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a19b457-b77e-43ad-aea9-f09eed3a2a7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:04:47.459350 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a19b457-b77e-43ad-aea9-f09eed3a2a7b admin admin] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:04:47.459641 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a19b457-b77e-43ad-aea9-f09eed3a2a7b admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:04:47.460452 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a19b457-b77e-43ad-aea9-f09eed3a2a7b admin admin] PATH_INFO: `/v3/projects` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:04:47.463879 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a19b457-b77e-43ad-aea9-f09eed3a2a7b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:04:47.470847 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a19b457-b77e-43ad-aea9-f09eed3a2a7b admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:04:47.472243 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db157c0-8290-46ee-b309-231aa216bc47 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:04:47.487833 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c1409b-2832-4f72-9438-8c4593de6b97 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f4b4a662db049ecacc865ecff10bdb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e30ccf0c1204727b786876c141dbcd2', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:04:47.488841 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c1409b-2832-4f72-9438-8c4593de6b97 service nova] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:04:47.489203 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c1409b-2832-4f72-9438-8c4593de6b97 service nova] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:04:47.489481 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c1409b-2832-4f72-9438-8c4593de6b97 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:04:47.494017 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-94c1409b-2832-4f72-9438-8c4593de6b97 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:04:47.530106 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c1409b-2832-4f72-9438-8c4593de6b97 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:04:47.534589 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c1409b-2832-4f72-9438-8c4593de6b97 service nova] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:04:47.537589 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 846/1703] 10.222.0.40 () {60 vars in 1153 bytes} [Fri Aug 26 11:04:47 2022] POST /identity/v3/projects => generated 401 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 11:04:47.542481 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a19b457-b77e-43ad-aea9-f09eed3a2a7b admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:04:47.590957 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 11:04:47.612202 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 858/1704] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 11:04:47 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 11:04:47.744631 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a19b457-b77e-43ad-aea9-f09eed3a2a7b admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:04:47.771087 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873715ac-813c-4f7b-b44b-40fee9e33100 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:04:47.771872 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873715ac-813c-4f7b-b44b-40fee9e33100 admin admin] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:04:47.782466 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873715ac-813c-4f7b-b44b-40fee9e33100 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:04:47.783254 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873715ac-813c-4f7b-b44b-40fee9e33100 admin admin] PATH_INFO: `/v3/users` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:04:47.784565 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-873715ac-813c-4f7b-b44b-40fee9e33100 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:04:47.789936 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-873715ac-813c-4f7b-b44b-40fee9e33100 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:04:47.810929 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c1409b-2832-4f72-9438-8c4593de6b97 service nova] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:04:47.813307 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-94c1409b-2832-4f72-9438-8c4593de6b97 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:04:47.893243 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 847/1705] 10.222.0.40 () {60 vars in 1144 bytes} [Fri Aug 26 11:04:47 2022] POST /identity/v3/users => generated 301 bytes in 353 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 26 11:04:47.894228 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873715ac-813c-4f7b-b44b-40fee9e33100 admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:04:47.935958 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873715ac-813c-4f7b-b44b-40fee9e33100 admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:04:47.948118 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d22ae54-5321-4edd-9c6c-cb59f7056255 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:04:47.949023 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d22ae54-5321-4edd-9c6c-cb59f7056255 admin admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:04:47.949465 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d22ae54-5321-4edd-9c6c-cb59f7056255 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:04:47.950558 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d22ae54-5321-4edd-9c6c-cb59f7056255 admin admin] PATH_INFO: `/v3/roles` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:04:47.952137 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d22ae54-5321-4edd-9c6c-cb59f7056255 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:04:47.954451 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d22ae54-5321-4edd-9c6c-cb59f7056255 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:04:47.974103 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 848/1706] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 26 11:04:47 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 11:04:47.978277 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d22ae54-5321-4edd-9c6c-cb59f7056255 admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:04:47.994780 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d22ae54-5321-4edd-9c6c-cb59f7056255 admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:04:48.003344 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ca346b-0d01-432e-81c0-3c7f74bdc135 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:04:48.004513 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ca346b-0d01-432e-81c0-3c7f74bdc135 admin admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:04:48.005639 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ca346b-0d01-432e-81c0-3c7f74bdc135 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:04:48.006112 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ca346b-0d01-432e-81c0-3c7f74bdc135 admin admin] PATH_INFO: `/v3/roles` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:04:48.007543 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ca346b-0d01-432e-81c0-3c7f74bdc135 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:04:48.012891 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ca346b-0d01-432e-81c0-3c7f74bdc135 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:04:48.032550 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 849/1707] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 26 11:04:47 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 11:04:48.037293 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ca346b-0d01-432e-81c0-3c7f74bdc135 admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:04:48.039912 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-b3ca346b-0d01-432e-81c0-3c7f74bdc135 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:04:48.065837 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ca346b-0d01-432e-81c0-3c7f74bdc135 admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:04:48.070205 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c1409b-2832-4f72-9438-8c4593de6b97 service nova] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:04:48.080461 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ca8562-c855-480f-b1c7-4078723c64fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:04:48.081176 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ca8562-c855-480f-b1c7-4078723c64fd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:04:48.081461 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ca8562-c855-480f-b1c7-4078723c64fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:04:48.081733 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ca8562-c855-480f-b1c7-4078723c64fd admin admin] PATH_INFO: `/v3/users/f1e8970431174986a95dd75a605db777` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:04:48.086452 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ca8562-c855-480f-b1c7-4078723c64fd admin admin] RBAC: Authorizing `identity:delete_user(user_id=f1e8970431174986a95dd75a605db777)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:04:48.093857 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ca8562-c855-480f-b1c7-4078723c64fd admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:04:48.246768 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 11:04:48.264123 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-c0ca8562-c855-480f-b1c7-4078723c64fd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f1e8970431174986a95dd75a605db777'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:04:48.266056 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee011cc-8614-4d69-9b04-c39dc14c28a3 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f29c5dfed31f42a9b5cbc0cd72e9fdc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e30ccf0c1204727b786876c141dbcd2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:04:48.267186 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee011cc-8614-4d69-9b04-c39dc14c28a3 service placement] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:04:48.267625 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee011cc-8614-4d69-9b04-c39dc14c28a3 service placement] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:04:48.268050 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee011cc-8614-4d69-9b04-c39dc14c28a3 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:04:48.272258 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-fee011cc-8614-4d69-9b04-c39dc14c28a3 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:04:48.295125 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-c0ca8562-c855-480f-b1c7-4078723c64fd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f1e8970431174986a95dd75a605db777'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:04:48.297206 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-c0ca8562-c855-480f-b1c7-4078723c64fd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f1e8970431174986a95dd75a605db777'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:04:48.300045 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee011cc-8614-4d69-9b04-c39dc14c28a3 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:04:48.304420 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee011cc-8614-4d69-9b04-c39dc14c28a3 service placement] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:04:48.338281 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-c0ca8562-c855-480f-b1c7-4078723c64fd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f1e8970431174986a95dd75a605db777'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:04:48.339090 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-c0ca8562-c855-480f-b1c7-4078723c64fd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f1e8970431174986a95dd75a605db777'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:04:48.348513 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 859/1708] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 11:04:47 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 538 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 11:04:48.353504 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee011cc-8614-4d69-9b04-c39dc14c28a3 service placement] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:04:48.380005 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 850/1709] 10.222.0.40 () {58 vars in 1224 bytes} [Fri Aug 26 11:04:48 2022] DELETE /identity/v3/users/f1e8970431174986a95dd75a605db777 => generated 0 bytes in 344 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 11:04:48.384813 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b6ad99-50ed-459a-ae46-b7968979b676 None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:04:48.385110 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b6ad99-50ed-459a-ae46-b7968979b676 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:04:48.385383 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b6ad99-50ed-459a-ae46-b7968979b676 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:04:48.388067 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee011cc-8614-4d69-9b04-c39dc14c28a3 service placement] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:04:48.450143 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-15b6ad99-50ed-459a-ae46-b7968979b676 None None] MFA Rules not processed for user `d6dbbabdd88c4c088bc91e6c43531318`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 11:04:48.455157 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-15b6ad99-50ed-459a-ae46-b7968979b676 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:04:48.566618 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc68c287-2de7-49f2-8f6d-020470a23f9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:04:48.585928 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc68c287-2de7-49f2-8f6d-020470a23f9f admin admin] REQUEST_METHOD: `PUT` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:04:48.588091 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc68c287-2de7-49f2-8f6d-020470a23f9f admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:04:48.588466 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc68c287-2de7-49f2-8f6d-020470a23f9f admin admin] PATH_INFO: `/v3/projects/9f4328d0c54044d3b1cb751de689fdeb/users/8cd0095e694847b49d94bd5e2f80fd57/roles/23e174ce743b46eabe618c212691da55` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:04:48.642021 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc68c287-2de7-49f2-8f6d-020470a23f9f admin admin] RBAC: Authorizing `identity:create_grant(project_id=9f4328d0c54044d3b1cb751de689fdeb, user_id=8cd0095e694847b49d94bd5e2f80fd57, role_id=23e174ce743b46eabe618c212691da55)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:04:48.645219 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc68c287-2de7-49f2-8f6d-020470a23f9f admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:04:48.662434 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 11:04:48.688400 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 851/1710] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 11:04:48 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 309 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 11:04:48.692466 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91bdb3b0-e457-49d4-9d50-fc44e1dd0228 None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:04:48.692466 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91bdb3b0-e457-49d4-9d50-fc44e1dd0228 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:04:48.692466 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91bdb3b0-e457-49d4-9d50-fc44e1dd0228 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:04:48.707232 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 860/1711] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 26 11:04:48 2022] PUT /identity/v3/projects/9f4328d0c54044d3b1cb751de689fdeb/users/8cd0095e694847b49d94bd5e2f80fd57/roles/23e174ce743b46eabe618c212691da55 => generated 0 bytes in 358 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 11:04:48.718500 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8692cfff-b0fc-47bb-95d8-f638c16003c7 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:04:48.719071 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8692cfff-b0fc-47bb-95d8-f638c16003c7 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:04:48.720027 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8692cfff-b0fc-47bb-95d8-f638c16003c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:04:48.730403 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-91bdb3b0-e457-49d4-9d50-fc44e1dd0228 None None] MFA Rules not processed for user `ca155a48e2024c32af3fd2e19e5fac11`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 11:04:48.731662 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-91bdb3b0-e457-49d4-9d50-fc44e1dd0228 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:04:48.794965 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-8692cfff-b0fc-47bb-95d8-f638c16003c7 None None] MFA Rules not processed for user `8cd0095e694847b49d94bd5e2f80fd57`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 11:04:48.800062 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-8692cfff-b0fc-47bb-95d8-f638c16003c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:04:48.884684 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 852/1712] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 11:04:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 11:04:48.962287 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 861/1713] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 11:04:48 2022] POST /identity/v3/auth/tokens => generated 3278 bytes in 241 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 11:04:48.963890 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91bdb3b0-e457-49d4-9d50-fc44e1dd0228 None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:04:48.965143 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8692cfff-b0fc-47bb-95d8-f638c16003c7 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:04:48.966819 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-91bdb3b0-e457-49d4-9d50-fc44e1dd0228 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:04:48.967169 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-8692cfff-b0fc-47bb-95d8-f638c16003c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:04:48.993297 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91bdb3b0-e457-49d4-9d50-fc44e1dd0228 None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:04:49.019650 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6b5438-9db8-4972-b552-692e73b60ea0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:04:49.020287 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6b5438-9db8-4972-b552-692e73b60ea0 admin admin] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:04:49.020561 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6b5438-9db8-4972-b552-692e73b60ea0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:04:49.020827 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6b5438-9db8-4972-b552-692e73b60ea0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:04:49.021459 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe6b5438-9db8-4972-b552-692e73b60ea0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:04:49.026250 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe6b5438-9db8-4972-b552-692e73b60ea0 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:04:49.055507 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8692cfff-b0fc-47bb-95d8-f638c16003c7 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:04:49.059841 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a776767-5fbe-4ce8-9b21-cb092db89588 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91da2acd2534423091537e1ba068d2b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e30ccf0c1204727b786876c141dbcd2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:04:49.060470 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a776767-5fbe-4ce8-9b21-cb092db89588 service neutron] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:04:49.060805 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a776767-5fbe-4ce8-9b21-cb092db89588 service neutron] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:04:49.061084 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a776767-5fbe-4ce8-9b21-cb092db89588 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:04:49.081320 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a776767-5fbe-4ce8-9b21-cb092db89588 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:04:49.084031 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a776767-5fbe-4ce8-9b21-cb092db89588 service neutron] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:04:49.100087 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 862/1714] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 11:04:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 26 11:04:49.121635 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 853/1715] 10.222.0.40 () {60 vars in 1153 bytes} [Fri Aug 26 11:04:48 2022] POST /identity/v3/projects => generated 401 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 11:04:49.123599 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a776767-5fbe-4ce8-9b21-cb092db89588 service neutron] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:04:49.139861 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 11:04:49.144834 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a776767-5fbe-4ce8-9b21-cb092db89588 service neutron] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:04:49.297322 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcdf8fc6-9bab-4324-a211-e8327d4c827a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:04:49.297921 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdf8fc6-9bab-4324-a211-e8327d4c827a admin admin] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:04:49.298243 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdf8fc6-9bab-4324-a211-e8327d4c827a admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:04:49.298520 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdf8fc6-9bab-4324-a211-e8327d4c827a admin admin] PATH_INFO: `/v3/users` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:04:49.299269 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcdf8fc6-9bab-4324-a211-e8327d4c827a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:04:49.313340 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcdf8fc6-9bab-4324-a211-e8327d4c827a admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:04:49.365161 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 863/1716] 10.222.0.40 () {60 vars in 1144 bytes} [Fri Aug 26 11:04:49 2022] POST /identity/v3/users => generated 307 bytes in 243 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 11:04:49.368417 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6b5438-9db8-4972-b552-692e73b60ea0 admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:04:49.385996 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6b5438-9db8-4972-b552-692e73b60ea0 admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:04:49.397143 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a57948-d4a2-43c7-aa1e-1c26e0688d5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:04:49.399452 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a57948-d4a2-43c7-aa1e-1c26e0688d5b admin admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:04:49.399742 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a57948-d4a2-43c7-aa1e-1c26e0688d5b admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:04:49.401086 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a57948-d4a2-43c7-aa1e-1c26e0688d5b admin admin] PATH_INFO: `/v3/roles` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:04:49.401829 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a57948-d4a2-43c7-aa1e-1c26e0688d5b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:04:49.404204 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a57948-d4a2-43c7-aa1e-1c26e0688d5b admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:04:49.420321 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 854/1717] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 26 11:04:49 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 11:04:49.421840 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcdf8fc6-9bab-4324-a211-e8327d4c827a admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:04:49.438183 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcdf8fc6-9bab-4324-a211-e8327d4c827a admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:04:49.452233 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54105713-dc8c-49c1-b622-323f25925a0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:04:49.466304 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d868cf8f-4cba-4853-aa61-8a1016016f11 None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:04:49.466782 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d868cf8f-4cba-4853-aa61-8a1016016f11 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:04:49.467212 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d868cf8f-4cba-4853-aa61-8a1016016f11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:04:49.477144 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54105713-dc8c-49c1-b622-323f25925a0a admin admin] REQUEST_METHOD: `PUT` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:04:49.477536 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54105713-dc8c-49c1-b622-323f25925a0a admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:04:49.477826 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54105713-dc8c-49c1-b622-323f25925a0a admin admin] PATH_INFO: `/v3/projects/7dce33deb5d04b26b1ea19ffa7a89555/users/53988ed352744aabbcc45d19ed15db6c/roles/1f97e4602e2e4d6498913a81f9ae8d6a` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:04:49.530182 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54105713-dc8c-49c1-b622-323f25925a0a admin admin] RBAC: Authorizing `identity:create_grant(project_id=7dce33deb5d04b26b1ea19ffa7a89555, user_id=53988ed352744aabbcc45d19ed15db6c, role_id=1f97e4602e2e4d6498913a81f9ae8d6a)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:04:49.539726 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-d868cf8f-4cba-4853-aa61-8a1016016f11 None None] MFA Rules not processed for user `aedac3f62dd94f87b968f6c71d68a0a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 11:04:49.541903 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54105713-dc8c-49c1-b622-323f25925a0a admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:04:49.543436 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-d868cf8f-4cba-4853-aa61-8a1016016f11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:04:49.581907 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 864/1718] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 26 11:04:49 2022] PUT /identity/v3/projects/7dce33deb5d04b26b1ea19ffa7a89555/users/53988ed352744aabbcc45d19ed15db6c/roles/1f97e4602e2e4d6498913a81f9ae8d6a => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 11:04:49.583316 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54105713-dc8c-49c1-b622-323f25925a0a admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:04:49.782406 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 855/1719] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 11:04:49 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 317 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 11:04:49.839891 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54105713-dc8c-49c1-b622-323f25925a0a admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:04:49.854789 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61bbab4e-2f3a-4d66-b58c-877a90ef98c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:04:49.855653 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bbab4e-2f3a-4d66-b58c-877a90ef98c3 admin admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:04:49.856799 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bbab4e-2f3a-4d66-b58c-877a90ef98c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:04:49.858355 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bbab4e-2f3a-4d66-b58c-877a90ef98c3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:04:49.859285 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61bbab4e-2f3a-4d66-b58c-877a90ef98c3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:04:49.867682 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61bbab4e-2f3a-4d66-b58c-877a90ef98c3 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:04:49.899530 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 865/1720] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 26 11:04:49 2022] GET /identity/v3/roles => generated 1420 bytes in 316 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 11:04:49.899896 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d868cf8f-4cba-4853-aa61-8a1016016f11 None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:04:49.923737 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d868cf8f-4cba-4853-aa61-8a1016016f11 None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:04:49.947585 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4709faa1-640a-4a1b-88f5-9798dfe81239 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:04:49.952038 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4709faa1-640a-4a1b-88f5-9798dfe81239 admin admin] REQUEST_METHOD: `PUT` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:04:49.955460 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4709faa1-640a-4a1b-88f5-9798dfe81239 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:04:49.956174 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4709faa1-640a-4a1b-88f5-9798dfe81239 admin admin] PATH_INFO: `/v3/projects/7dce33deb5d04b26b1ea19ffa7a89555/users/53988ed352744aabbcc45d19ed15db6c/roles/1f97e4602e2e4d6498913a81f9ae8d6a` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:04:49.962283 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4709faa1-640a-4a1b-88f5-9798dfe81239 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7dce33deb5d04b26b1ea19ffa7a89555, user_id=53988ed352744aabbcc45d19ed15db6c, role_id=1f97e4602e2e4d6498913a81f9ae8d6a)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:04:49.968559 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4709faa1-640a-4a1b-88f5-9798dfe81239 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:04:49.988934 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 856/1721] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 26 11:04:49 2022] PUT /identity/v3/projects/7dce33deb5d04b26b1ea19ffa7a89555/users/53988ed352744aabbcc45d19ed15db6c/roles/1f97e4602e2e4d6498913a81f9ae8d6a => generated 0 bytes in 90 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 26 11:04:50.000151 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37af89f5-4a5e-4a61-bf96-ef3f7fc0837f None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:04:50.002509 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37af89f5-4a5e-4a61-bf96-ef3f7fc0837f None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:04:50.003167 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37af89f5-4a5e-4a61-bf96-ef3f7fc0837f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:04:50.070139 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1997c6c8-0dca-4206-b8dd-588a7918bd98 None None] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:04:50.070795 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1997c6c8-0dca-4206-b8dd-588a7918bd98 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:04:50.071170 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1997c6c8-0dca-4206-b8dd-588a7918bd98 None None] PATH_INFO: `/` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:04:50.074744 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 857/1722] 10.222.0.40 () {54 vars in 912 bytes} [Fri Aug 26 11:04:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 11:04:50.075718 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-37af89f5-4a5e-4a61-bf96-ef3f7fc0837f None None] MFA Rules not processed for user `53988ed352744aabbcc45d19ed15db6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 11:04:50.076979 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-37af89f5-4a5e-4a61-bf96-ef3f7fc0837f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:04:50.082393 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e27ed8f-3368-4be2-838d-a72a3cf63b77 None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:04:50.082650 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e27ed8f-3368-4be2-838d-a72a3cf63b77 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:04:50.083051 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e27ed8f-3368-4be2-838d-a72a3cf63b77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:04:50.131559 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-5e27ed8f-3368-4be2-838d-a72a3cf63b77 None None] MFA Rules not processed for user `91da2acd2534423091537e1ba068d2b7`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 11:04:50.137373 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-5e27ed8f-3368-4be2-838d-a72a3cf63b77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:04:50.232486 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 11:04:50.266541 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 11:04:50.278174 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 866/1723] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 11:04:49 2022] POST /identity/v3/auth/tokens => generated 3345 bytes in 276 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 11:04:50.289115 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37af89f5-4a5e-4a61-bf96-ef3f7fc0837f None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:04:50.290611 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-37af89f5-4a5e-4a61-bf96-ef3f7fc0837f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:04:50.308021 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 858/1724] 10.222.0.40 () {58 vars in 1009 bytes} [Fri Aug 26 11:04:50 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 11:04:50.312070 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e27ed8f-3368-4be2-838d-a72a3cf63b77 None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:04:50.313610 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-5e27ed8f-3368-4be2-838d-a72a3cf63b77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:04:50.428874 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e27ed8f-3368-4be2-838d-a72a3cf63b77 None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:04:50.441167 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da82240d-d27d-4a60-ba72-981c972d401b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:04:50.442653 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da82240d-d27d-4a60-ba72-981c972d401b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:04:50.443326 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da82240d-d27d-4a60-ba72-981c972d401b admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:04:50.445504 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da82240d-d27d-4a60-ba72-981c972d401b admin admin] PATH_INFO: `/v3/projects/41e03cfe608f47beabce918524dfa2b6` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:04:50.451875 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da82240d-d27d-4a60-ba72-981c972d401b admin admin] RBAC: Authorizing `identity:delete_project(project_id=41e03cfe608f47beabce918524dfa2b6)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:04:50.454588 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da82240d-d27d-4a60-ba72-981c972d401b admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:04:50.468080 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37af89f5-4a5e-4a61-bf96-ef3f7fc0837f None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:04:50.482637 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dadaf004-3fd2-48fd-96db-3a11c5f947cd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c277338a7f814cc6b01c5419775031bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e30ccf0c1204727b786876c141dbcd2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:04:50.483293 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dadaf004-3fd2-48fd-96db-3a11c5f947cd service cinder] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:04:50.483570 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dadaf004-3fd2-48fd-96db-3a11c5f947cd service cinder] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:04:50.483836 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dadaf004-3fd2-48fd-96db-3a11c5f947cd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:04:50.517565 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dadaf004-3fd2-48fd-96db-3a11c5f947cd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:04:50.519955 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dadaf004-3fd2-48fd-96db-3a11c5f947cd service cinder] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:04:50.580006 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-da82240d-d27d-4a60-ba72-981c972d401b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '41e03cfe608f47beabce918524dfa2b6', 'request_id': 'req-da82240d-d27d-4a60-ba72-981c972d401b'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:04:50.580416 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 867/1725] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 11:04:50 2022] GET /identity/v3/auth/tokens => generated 3345 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 26 11:04:50.581562 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dadaf004-3fd2-48fd-96db-3a11c5f947cd service cinder] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:04:50.586183 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-dadaf004-3fd2-48fd-96db-3a11c5f947cd service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:04:50.605448 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-da82240d-d27d-4a60-ba72-981c972d401b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '41e03cfe608f47beabce918524dfa2b6', 'request_id': 'req-da82240d-d27d-4a60-ba72-981c972d401b'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:04:50.612507 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-da82240d-d27d-4a60-ba72-981c972d401b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '41e03cfe608f47beabce918524dfa2b6', 'request_id': 'req-da82240d-d27d-4a60-ba72-981c972d401b'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:04:50.617317 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dadaf004-3fd2-48fd-96db-3a11c5f947cd service cinder] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:04:50.634906 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-da82240d-d27d-4a60-ba72-981c972d401b admin admin] The token cache is being invalidate because project 41e03cfe608f47beabce918524dfa2b6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81423) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 11:04:50.635584 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-da82240d-d27d-4a60-ba72-981c972d401b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:04:50.638129 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-da82240d-d27d-4a60-ba72-981c972d401b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:04:50.641792 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 859/1726] 10.222.0.40 () {58 vars in 1233 bytes} [Fri Aug 26 11:04:50 2022] DELETE /identity/v3/projects/41e03cfe608f47beabce918524dfa2b6 => generated 0 bytes in 330 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 11:04:50.643376 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e26165-44b3-47c3-b1bd-b508741910a8 None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:04:50.643853 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e26165-44b3-47c3-b1bd-b508741910a8 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:04:50.644352 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e26165-44b3-47c3-b1bd-b508741910a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:04:50.707058 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-e7e26165-44b3-47c3-b1bd-b508741910a8 None None] MFA Rules not processed for user `163e8be86d5640f88893c50e6f3a8af7`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 11:04:50.709641 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-e7e26165-44b3-47c3-b1bd-b508741910a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:04:50.732328 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a1d9ab-4c48-4bed-b9dc-0107f5e69a9d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91da2acd2534423091537e1ba068d2b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e30ccf0c1204727b786876c141dbcd2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:04:50.733410 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a1d9ab-4c48-4bed-b9dc-0107f5e69a9d service neutron] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:04:50.733911 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a1d9ab-4c48-4bed-b9dc-0107f5e69a9d service neutron] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:04:50.734500 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a1d9ab-4c48-4bed-b9dc-0107f5e69a9d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:04:50.737372 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-e4a1d9ab-4c48-4bed-b9dc-0107f5e69a9d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:04:50.761471 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4a1d9ab-4c48-4bed-b9dc-0107f5e69a9d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:04:50.769650 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4a1d9ab-4c48-4bed-b9dc-0107f5e69a9d service neutron] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:04:50.797728 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 868/1727] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 11:04:50 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 217 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 11:04:50.798634 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a1d9ab-4c48-4bed-b9dc-0107f5e69a9d service neutron] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:04:50.800004 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-e4a1d9ab-4c48-4bed-b9dc-0107f5e69a9d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:04:50.872628 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 860/1728] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 11:04:50 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 230 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 11:04:50.883114 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe14dce-e74a-4021-a1fa-243f6a41b96c None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:04:50.883418 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe14dce-e74a-4021-a1fa-243f6a41b96c None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:04:50.883672 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe14dce-e74a-4021-a1fa-243f6a41b96c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:04:50.929246 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-7fe14dce-e74a-4021-a1fa-243f6a41b96c None None] MFA Rules not processed for user `ca155a48e2024c32af3fd2e19e5fac11`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 11:04:50.930550 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-7fe14dce-e74a-4021-a1fa-243f6a41b96c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:04:50.932659 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a1d9ab-4c48-4bed-b9dc-0107f5e69a9d service neutron] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:04:50.937648 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a5aac7-46c1-4038-bd7b-610be95db397 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:04:50.939120 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a5aac7-46c1-4038-bd7b-610be95db397 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:04:50.939570 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a5aac7-46c1-4038-bd7b-610be95db397 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:04:50.940052 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a5aac7-46c1-4038-bd7b-610be95db397 admin admin] PATH_INFO: `/v3/projects/b37ac3b5e4114f02b08a4b2c8c3b60f8` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:04:50.941833 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a5aac7-46c1-4038-bd7b-610be95db397 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b37ac3b5e4114f02b08a4b2c8c3b60f8)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:04:50.944331 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a5aac7-46c1-4038-bd7b-610be95db397 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:04:50.952636 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 861/1729] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 11:04:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 11:04:50.954211 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe14dce-e74a-4021-a1fa-243f6a41b96c None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:04:50.955636 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-7fe14dce-e74a-4021-a1fa-243f6a41b96c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:04:50.965326 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 11:04:51.061378 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 11:04:51.071961 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe14dce-e74a-4021-a1fa-243f6a41b96c None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:04:51.073948 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-26a5aac7-46c1-4038-bd7b-610be95db397 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b37ac3b5e4114f02b08a4b2c8c3b60f8', 'request_id': 'req-26a5aac7-46c1-4038-bd7b-610be95db397'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:04:51.088831 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-26a5aac7-46c1-4038-bd7b-610be95db397 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b37ac3b5e4114f02b08a4b2c8c3b60f8', 'request_id': 'req-26a5aac7-46c1-4038-bd7b-610be95db397'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:04:51.099111 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-26a5aac7-46c1-4038-bd7b-610be95db397 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b37ac3b5e4114f02b08a4b2c8c3b60f8', 'request_id': 'req-26a5aac7-46c1-4038-bd7b-610be95db397'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:04:51.124334 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-26a5aac7-46c1-4038-bd7b-610be95db397 admin admin] The token cache is being invalidate because project b37ac3b5e4114f02b08a4b2c8c3b60f8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81424) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 11:04:51.124675 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-26a5aac7-46c1-4038-bd7b-610be95db397 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:04:51.125407 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-26a5aac7-46c1-4038-bd7b-610be95db397 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:04:51.129466 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 869/1730] 10.222.0.40 () {58 vars in 1233 bytes} [Fri Aug 26 11:04:50 2022] DELETE /identity/v3/projects/b37ac3b5e4114f02b08a4b2c8c3b60f8 => generated 0 bytes in 331 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 11:04:51.131093 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6bbe21-047c-40b9-b694-c9075096e9de None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:04:51.131329 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6bbe21-047c-40b9-b694-c9075096e9de None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:04:51.132386 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6bbe21-047c-40b9-b694-c9075096e9de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:04:51.169704 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-1c6bbe21-047c-40b9-b694-c9075096e9de None None] MFA Rules not processed for user `ca155a48e2024c32af3fd2e19e5fac11`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 11:04:51.171159 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-1c6bbe21-047c-40b9-b694-c9075096e9de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:04:51.176569 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8176f27-b149-4d6c-ad64-4b90ce2bca8b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c277338a7f814cc6b01c5419775031bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e30ccf0c1204727b786876c141dbcd2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:04:51.178407 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8176f27-b149-4d6c-ad64-4b90ce2bca8b service cinder] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:04:51.178407 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8176f27-b149-4d6c-ad64-4b90ce2bca8b service cinder] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:04:51.184058 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8176f27-b149-4d6c-ad64-4b90ce2bca8b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:04:51.187250 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-e8176f27-b149-4d6c-ad64-4b90ce2bca8b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:04:51.278073 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 870/1731] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 11:04:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 11:04:51.278073 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6bbe21-047c-40b9-b694-c9075096e9de None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:04:51.279441 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-1c6bbe21-047c-40b9-b694-c9075096e9de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:04:51.296753 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6bbe21-047c-40b9-b694-c9075096e9de None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:04:51.309016 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9158e8d-38e1-46a4-a14c-a30d13c591ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:04:51.309955 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9158e8d-38e1-46a4-a14c-a30d13c591ff admin admin] REQUEST_METHOD: `DELETE` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:04:51.310248 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9158e8d-38e1-46a4-a14c-a30d13c591ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:04:51.310524 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9158e8d-38e1-46a4-a14c-a30d13c591ff admin admin] PATH_INFO: `/v3/projects/16700221d9794d5bab7029250b3e39ee` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:04:51.313277 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9158e8d-38e1-46a4-a14c-a30d13c591ff admin admin] RBAC: Authorizing `identity:delete_project(project_id=16700221d9794d5bab7029250b3e39ee)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:04:51.315256 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9158e8d-38e1-46a4-a14c-a30d13c591ff admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:04:51.332590 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8176f27-b149-4d6c-ad64-4b90ce2bca8b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:04:51.334517 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8176f27-b149-4d6c-ad64-4b90ce2bca8b service cinder] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:04:51.360331 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 862/1732] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 11:04:50 2022] GET /identity/v3/auth/tokens => generated 3278 bytes in 407 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 11:04:51.365746 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8176f27-b149-4d6c-ad64-4b90ce2bca8b service cinder] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:04:51.370761 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-e8176f27-b149-4d6c-ad64-4b90ce2bca8b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:04:51.399714 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-b9158e8d-38e1-46a4-a14c-a30d13c591ff admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '16700221d9794d5bab7029250b3e39ee', 'request_id': 'req-b9158e8d-38e1-46a4-a14c-a30d13c591ff'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:04:51.415147 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8176f27-b149-4d6c-ad64-4b90ce2bca8b service cinder] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:04:51.417467 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-b9158e8d-38e1-46a4-a14c-a30d13c591ff admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '16700221d9794d5bab7029250b3e39ee', 'request_id': 'req-b9158e8d-38e1-46a4-a14c-a30d13c591ff'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:04:51.425242 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-b9158e8d-38e1-46a4-a14c-a30d13c591ff admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '16700221d9794d5bab7029250b3e39ee', 'request_id': 'req-b9158e8d-38e1-46a4-a14c-a30d13c591ff'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:04:51.447046 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-b9158e8d-38e1-46a4-a14c-a30d13c591ff admin admin] The token cache is being invalidate because project 16700221d9794d5bab7029250b3e39ee was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81424) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 11:04:51.447452 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-b9158e8d-38e1-46a4-a14c-a30d13c591ff admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:04:51.461922 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-b9158e8d-38e1-46a4-a14c-a30d13c591ff admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:04:51.470566 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 871/1733] 10.222.0.40 () {58 vars in 1233 bytes} [Fri Aug 26 11:04:51 2022] DELETE /identity/v3/projects/16700221d9794d5bab7029250b3e39ee => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 11:04:51.471625 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9158e8d-38e1-46a4-a14c-a30d13c591ff admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:04:51.480279 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-b9158e8d-38e1-46a4-a14c-a30d13c591ff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:04:51.586901 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=87,keystone:DELETE=11,keystone:INSERT=2 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 11:04:51.623894 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda28e40-7186-4ca3-a77c-950695b379dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:04:51.624575 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda28e40-7186-4ca3-a77c-950695b379dc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:04:51.628039 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda28e40-7186-4ca3-a77c-950695b379dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:04:51.628211 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda28e40-7186-4ca3-a77c-950695b379dc admin admin] PATH_INFO: `/v3/users/e4404bb8f8b942618c3b2992bcbe3d71` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:04:51.629036 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9158e8d-38e1-46a4-a14c-a30d13c591ff admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:04:51.634543 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda28e40-7186-4ca3-a77c-950695b379dc admin admin] RBAC: Authorizing `identity:delete_user(user_id=e4404bb8f8b942618c3b2992bcbe3d71)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:04:51.636515 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda28e40-7186-4ca3-a77c-950695b379dc admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:04:51.646169 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431baa46-281e-449a-b6e2-de3d9d5f7e62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:04:51.646169 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431baa46-281e-449a-b6e2-de3d9d5f7e62 admin admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:04:51.646169 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431baa46-281e-449a-b6e2-de3d9d5f7e62 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:04:51.647168 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431baa46-281e-449a-b6e2-de3d9d5f7e62 admin admin] PATH_INFO: `/v3/domains` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:04:51.652754 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-431baa46-281e-449a-b6e2-de3d9d5f7e62 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 11:04:51.652754 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-431baa46-281e-449a-b6e2-de3d9d5f7e62 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:04:51.654479 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-431baa46-281e-449a-b6e2-de3d9d5f7e62 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:04:51.695611 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 11:04:51.699324 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 872/1734] 10.222.0.40 () {58 vars in 1153 bytes} [Fri Aug 26 11:04:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 11:04:51.712831 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431baa46-281e-449a-b6e2-de3d9d5f7e62 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:04:51.721325 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-431baa46-281e-449a-b6e2-de3d9d5f7e62 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:04:51.786966 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431baa46-281e-449a-b6e2-de3d9d5f7e62 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:04:51.808190 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2b8652-befe-4dfa-9a30-d613e835415c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:04:51.809890 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2b8652-befe-4dfa-9a30-d613e835415c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:04:51.810105 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2b8652-befe-4dfa-9a30-d613e835415c admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:04:51.815140 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2b8652-befe-4dfa-9a30-d613e835415c admin admin] PATH_INFO: `/v3/users/65fd1dae078f41138e6c5a958e0972fc` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:04:51.817396 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc2b8652-befe-4dfa-9a30-d613e835415c admin admin] RBAC: Authorizing `identity:delete_user(user_id=65fd1dae078f41138e6c5a958e0972fc)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:04:51.823474 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc2b8652-befe-4dfa-9a30-d613e835415c admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:04:51.828755 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-cda28e40-7186-4ca3-a77c-950695b379dc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e4404bb8f8b942618c3b2992bcbe3d71'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:04:51.861551 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-cda28e40-7186-4ca3-a77c-950695b379dc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e4404bb8f8b942618c3b2992bcbe3d71'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:04:51.862407 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-cda28e40-7186-4ca3-a77c-950695b379dc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e4404bb8f8b942618c3b2992bcbe3d71'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:04:51.912728 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-cda28e40-7186-4ca3-a77c-950695b379dc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e4404bb8f8b942618c3b2992bcbe3d71'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:04:51.913780 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-cda28e40-7186-4ca3-a77c-950695b379dc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e4404bb8f8b942618c3b2992bcbe3d71'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:04:51.964997 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 863/1735] 10.222.0.40 () {58 vars in 1224 bytes} [Fri Aug 26 11:04:51 2022] DELETE /identity/v3/users/e4404bb8f8b942618c3b2992bcbe3d71 => generated 0 bytes in 604 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 11:04:51.966682 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda28e40-7186-4ca3-a77c-950695b379dc admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:04:51.967288 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-cc2b8652-befe-4dfa-9a30-d613e835415c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '65fd1dae078f41138e6c5a958e0972fc'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:04:51.969935 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-cda28e40-7186-4ca3-a77c-950695b379dc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:04:51.970167 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-cc2b8652-befe-4dfa-9a30-d613e835415c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '65fd1dae078f41138e6c5a958e0972fc'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:04:51.991700 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-cc2b8652-befe-4dfa-9a30-d613e835415c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '65fd1dae078f41138e6c5a958e0972fc'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:04:51.994248 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-cc2b8652-befe-4dfa-9a30-d613e835415c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '65fd1dae078f41138e6c5a958e0972fc'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:04:52.054089 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-cc2b8652-befe-4dfa-9a30-d613e835415c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '65fd1dae078f41138e6c5a958e0972fc'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:04:52.093824 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 873/1736] 10.222.0.40 () {58 vars in 1224 bytes} [Fri Aug 26 11:04:51 2022] DELETE /identity/v3/users/65fd1dae078f41138e6c5a958e0972fc => generated 0 bytes in 394 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 11:04:52.110533 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2b8652-befe-4dfa-9a30-d613e835415c admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:04:52.115420 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-cc2b8652-befe-4dfa-9a30-d613e835415c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:04:52.373392 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda28e40-7186-4ca3-a77c-950695b379dc admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:04:52.446279 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2b8652-befe-4dfa-9a30-d613e835415c admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:04:52.463867 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d89ca2a-2d66-440e-9434-397fb7e89a2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:04:52.464578 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d89ca2a-2d66-440e-9434-397fb7e89a2f admin admin] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:04:52.464861 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d89ca2a-2d66-440e-9434-397fb7e89a2f admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:04:52.465134 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d89ca2a-2d66-440e-9434-397fb7e89a2f admin admin] PATH_INFO: `/v3/projects` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:04:52.465773 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d89ca2a-2d66-440e-9434-397fb7e89a2f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:04:52.471189 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d89ca2a-2d66-440e-9434-397fb7e89a2f admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:04:52.557504 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba58ab97-c093-43fa-83f6-e3442c75ab15 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f4b4a662db049ecacc865ecff10bdb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e30ccf0c1204727b786876c141dbcd2', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:04:52.558401 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba58ab97-c093-43fa-83f6-e3442c75ab15 service nova] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:04:52.558908 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba58ab97-c093-43fa-83f6-e3442c75ab15 service nova] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:04:52.559385 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba58ab97-c093-43fa-83f6-e3442c75ab15 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:04:52.565060 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 874/1737] 10.222.0.40 () {60 vars in 1153 bytes} [Fri Aug 26 11:04:52 2022] POST /identity/v3/projects => generated 373 bytes in 470 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 11:04:52.566204 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d89ca2a-2d66-440e-9434-397fb7e89a2f admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:04:52.568635 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-ba58ab97-c093-43fa-83f6-e3442c75ab15 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:04:52.573474 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-2d89ca2a-2d66-440e-9434-397fb7e89a2f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:04:52.664780 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:DELETE=9,keystone:SELECT=90,keystone:INSERT=1 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 11:04:52.715375 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba58ab97-c093-43fa-83f6-e3442c75ab15 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:04:52.717638 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba58ab97-c093-43fa-83f6-e3442c75ab15 service nova] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:04:52.761305 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 864/1738] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 11:04:51 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 796 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 11:04:52.762383 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba58ab97-c093-43fa-83f6-e3442c75ab15 service nova] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:04:52.787300 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba58ab97-c093-43fa-83f6-e3442c75ab15 service nova] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:04:52.807430 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 11:04:52.940710 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d89ca2a-2d66-440e-9434-397fb7e89a2f admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:04:52.967128 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac4f792-bcec-4689-9ba1-4b15786507d3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f4b4a662db049ecacc865ecff10bdb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e30ccf0c1204727b786876c141dbcd2', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:04:52.968072 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac4f792-bcec-4689-9ba1-4b15786507d3 service nova] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:04:52.968629 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac4f792-bcec-4689-9ba1-4b15786507d3 service nova] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:04:52.969795 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4aba78-c898-46b1-b359-ba05eee2bfb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:04:52.970540 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4aba78-c898-46b1-b359-ba05eee2bfb9 admin admin] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:04:52.970740 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4aba78-c898-46b1-b359-ba05eee2bfb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:04:52.971019 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4aba78-c898-46b1-b359-ba05eee2bfb9 admin admin] PATH_INFO: `/v3/users` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:04:52.973112 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac4f792-bcec-4689-9ba1-4b15786507d3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:04:52.978803 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-2ac4f792-bcec-4689-9ba1-4b15786507d3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:04:52.993297 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c4aba78-c898-46b1-b359-ba05eee2bfb9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:04:52.999305 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c4aba78-c898-46b1-b359-ba05eee2bfb9 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:04:53.048909 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 865/1739] 10.222.0.40 () {60 vars in 1144 bytes} [Fri Aug 26 11:04:52 2022] POST /identity/v3/users => generated 287 bytes in 287 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 11:04:53.063488 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97d5bd0-2c47-4f6e-bf17-5be666be5c86 None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:04:53.065968 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97d5bd0-2c47-4f6e-bf17-5be666be5c86 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:04:53.067195 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97d5bd0-2c47-4f6e-bf17-5be666be5c86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:04:53.207283 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac4f792-bcec-4689-9ba1-4b15786507d3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:04:53.212525 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac4f792-bcec-4689-9ba1-4b15786507d3 service nova] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:04:53.259244 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 875/1740] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 11:04:52 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 692 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 11:04:53.264903 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac4f792-bcec-4689-9ba1-4b15786507d3 service nova] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:04:53.278432 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-d97d5bd0-2c47-4f6e-bf17-5be666be5c86 None None] MFA Rules not processed for user `50be12ccbd274a8792615e9dd10b61ca`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 11:04:53.279717 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-d97d5bd0-2c47-4f6e-bf17-5be666be5c86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:04:53.309312 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac4f792-bcec-4689-9ba1-4b15786507d3 service nova] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:04:53.325231 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32daa0c1-8e15-4128-9b0c-06bd58bf9ada admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:04:53.331212 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32daa0c1-8e15-4128-9b0c-06bd58bf9ada admin admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:04:53.331595 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32daa0c1-8e15-4128-9b0c-06bd58bf9ada admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:04:53.331885 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32daa0c1-8e15-4128-9b0c-06bd58bf9ada admin admin] PATH_INFO: `/v3/roles` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:04:53.332575 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32daa0c1-8e15-4128-9b0c-06bd58bf9ada admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:04:53.341084 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32daa0c1-8e15-4128-9b0c-06bd58bf9ada admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:04:53.395616 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 876/1741] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 26 11:04:53 2022] GET /identity/v3/roles => generated 1420 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 11:04:53.396582 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32daa0c1-8e15-4128-9b0c-06bd58bf9ada admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:04:53.446140 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32daa0c1-8e15-4128-9b0c-06bd58bf9ada admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:04:53.462030 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21c1a2a-9ca3-4c50-87bf-b5d487e35542 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:04:53.462668 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21c1a2a-9ca3-4c50-87bf-b5d487e35542 admin admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:04:53.462958 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21c1a2a-9ca3-4c50-87bf-b5d487e35542 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:04:53.463234 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21c1a2a-9ca3-4c50-87bf-b5d487e35542 admin admin] PATH_INFO: `/v3/roles` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:04:53.463892 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a21c1a2a-9ca3-4c50-87bf-b5d487e35542 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:04:53.472205 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a21c1a2a-9ca3-4c50-87bf-b5d487e35542 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:04:53.501283 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 877/1742] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 26 11:04:53 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 11:04:53.506465 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21c1a2a-9ca3-4c50-87bf-b5d487e35542 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:04:53.523206 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 866/1743] 10.222.0.40 () {58 vars in 1009 bytes} [Fri Aug 26 11:04:53 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 473 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 11:04:53.528024 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21c1a2a-9ca3-4c50-87bf-b5d487e35542 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:04:53.535088 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ebdafd9-08d7-4df3-9b75-79a0bdd25841 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:04:53.536082 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebdafd9-08d7-4df3-9b75-79a0bdd25841 admin admin] REQUEST_METHOD: `PUT` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:04:53.536515 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebdafd9-08d7-4df3-9b75-79a0bdd25841 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:04:53.536939 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebdafd9-08d7-4df3-9b75-79a0bdd25841 admin admin] PATH_INFO: `/v3/projects/8c339a176ac445689940c78db39d1370/users/fd154caf7c5d49e59e916666eca681b6/roles/23e174ce743b46eabe618c212691da55` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:04:53.632541 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ebdafd9-08d7-4df3-9b75-79a0bdd25841 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8c339a176ac445689940c78db39d1370, user_id=fd154caf7c5d49e59e916666eca681b6, role_id=23e174ce743b46eabe618c212691da55)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:04:53.644525 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ebdafd9-08d7-4df3-9b75-79a0bdd25841 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:04:53.694081 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 878/1744] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 26 11:04:53 2022] PUT /identity/v3/projects/8c339a176ac445689940c78db39d1370/users/fd154caf7c5d49e59e916666eca681b6/roles/23e174ce743b46eabe618c212691da55 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 11:04:53.705771 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc77f195-5771-42cb-a35a-c3456c57b52e None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:04:53.706254 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc77f195-5771-42cb-a35a-c3456c57b52e None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:04:53.706535 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc77f195-5771-42cb-a35a-c3456c57b52e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:04:53.781818 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-cc77f195-5771-42cb-a35a-c3456c57b52e None None] MFA Rules not processed for user `fd154caf7c5d49e59e916666eca681b6`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 11:04:53.783147 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-cc77f195-5771-42cb-a35a-c3456c57b52e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:04:53.812036 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 11:04:53.898390 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ebdafd9-08d7-4df3-9b75-79a0bdd25841 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:04:53.901345 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-8ebdafd9-08d7-4df3-9b75-79a0bdd25841 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:04:54.065029 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 867/1745] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 11:04:53 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 361 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 11:04:54.072222 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc77f195-5771-42cb-a35a-c3456c57b52e None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:04:54.245108 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ebdafd9-08d7-4df3-9b75-79a0bdd25841 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:04:54.276696 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd42c0e-8e39-40b0-adbb-018adba8e20a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6529bea296c9482a973ced5fe9f2212f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e30ccf0c1204727b786876c141dbcd2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:04:54.277685 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd42c0e-8e39-40b0-adbb-018adba8e20a service swift] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:04:54.280663 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd42c0e-8e39-40b0-adbb-018adba8e20a service swift] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:04:54.281128 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd42c0e-8e39-40b0-adbb-018adba8e20a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:04:54.292770 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc77f195-5771-42cb-a35a-c3456c57b52e None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:04:54.306613 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225c7072-9c24-4ad4-a0a2-0a93951cffd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:04:54.307283 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225c7072-9c24-4ad4-a0a2-0a93951cffd7 admin admin] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:04:54.307535 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225c7072-9c24-4ad4-a0a2-0a93951cffd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:04:54.308429 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225c7072-9c24-4ad4-a0a2-0a93951cffd7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:04:54.309093 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-225c7072-9c24-4ad4-a0a2-0a93951cffd7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:04:54.313396 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-225c7072-9c24-4ad4-a0a2-0a93951cffd7 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:04:54.338303 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 11:04:54.411331 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 868/1746] 10.222.0.40 () {60 vars in 1153 bytes} [Fri Aug 26 11:04:54 2022] POST /identity/v3/projects => generated 373 bytes in 342 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 11:04:54.413666 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225c7072-9c24-4ad4-a0a2-0a93951cffd7 admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:04:54.526583 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcd42c0e-8e39-40b0-adbb-018adba8e20a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:04:54.528615 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcd42c0e-8e39-40b0-adbb-018adba8e20a service swift] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:04:54.549323 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 879/1747] 10.222.0.40 () {60 vars in 1384 bytes} [Fri Aug 26 11:04:53 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 653 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 26 11:04:54.713125 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225c7072-9c24-4ad4-a0a2-0a93951cffd7 admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:04:54.726874 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d875e1b-11e7-4bf5-a705-8856260cdad5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:04:54.727530 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d875e1b-11e7-4bf5-a705-8856260cdad5 admin admin] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:04:54.727998 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d875e1b-11e7-4bf5-a705-8856260cdad5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:04:54.729022 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d875e1b-11e7-4bf5-a705-8856260cdad5 admin admin] PATH_INFO: `/v3/users` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:04:54.729702 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d875e1b-11e7-4bf5-a705-8856260cdad5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:04:54.735126 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d875e1b-11e7-4bf5-a705-8856260cdad5 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:04:54.810929 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 869/1748] 10.222.0.40 () {60 vars in 1144 bytes} [Fri Aug 26 11:04:54 2022] POST /identity/v3/users => generated 293 bytes in 397 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 11:04:54.812554 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd42c0e-8e39-40b0-adbb-018adba8e20a service swift] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:04:54.829879 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd42c0e-8e39-40b0-adbb-018adba8e20a service swift] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:04:54.860865 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642159c0-2cf3-4a9c-a2dd-08c89ab73c8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:04:54.861680 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642159c0-2cf3-4a9c-a2dd-08c89ab73c8a admin admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:04:54.862380 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642159c0-2cf3-4a9c-a2dd-08c89ab73c8a admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:04:54.862872 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642159c0-2cf3-4a9c-a2dd-08c89ab73c8a admin admin] PATH_INFO: `/v3/roles` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:04:54.863744 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-642159c0-2cf3-4a9c-a2dd-08c89ab73c8a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:04:54.878067 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-642159c0-2cf3-4a9c-a2dd-08c89ab73c8a admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:04:54.921384 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d875e1b-11e7-4bf5-a705-8856260cdad5 admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:04:54.921686 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 880/1749] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 26 11:04:54 2022] GET /identity/v3/roles => generated 1420 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 11:04:54.937911 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642159c0-2cf3-4a9c-a2dd-08c89ab73c8a admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:04:54.940057 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-642159c0-2cf3-4a9c-a2dd-08c89ab73c8a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:04:54.960933 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d875e1b-11e7-4bf5-a705-8856260cdad5 admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:04:54.981807 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2911fd-931a-4d6b-9ca9-b0353c243ade admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:04:54.983783 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642159c0-2cf3-4a9c-a2dd-08c89ab73c8a admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:04:54.985833 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2911fd-931a-4d6b-9ca9-b0353c243ade admin admin] REQUEST_METHOD: `PUT` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:04:54.986169 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2911fd-931a-4d6b-9ca9-b0353c243ade admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:04:54.986450 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2911fd-931a-4d6b-9ca9-b0353c243ade admin admin] PATH_INFO: `/v3/projects/2c5c7ed5385b4a2dafe1b0f3c4601f64/users/153feb43d9f346ed896b097bd4132526/roles/1f97e4602e2e4d6498913a81f9ae8d6a` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:04:55.006460 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f741f2-4264-4eca-901c-0dd2d84c680a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:04:55.010758 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f741f2-4264-4eca-901c-0dd2d84c680a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:04:55.016231 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f741f2-4264-4eca-901c-0dd2d84c680a admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:04:55.016645 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f741f2-4264-4eca-901c-0dd2d84c680a admin admin] PATH_INFO: `/v3/projects/1268423c21a043d38b2400a1ee2deec8` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:04:55.019320 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f741f2-4264-4eca-901c-0dd2d84c680a admin admin] RBAC: Authorizing `identity:delete_project(project_id=1268423c21a043d38b2400a1ee2deec8)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:04:55.026917 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f741f2-4264-4eca-901c-0dd2d84c680a admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:04:55.064356 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a2911fd-931a-4d6b-9ca9-b0353c243ade admin admin] RBAC: Authorizing `identity:create_grant(project_id=2c5c7ed5385b4a2dafe1b0f3c4601f64, user_id=153feb43d9f346ed896b097bd4132526, role_id=1f97e4602e2e4d6498913a81f9ae8d6a)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:04:55.094766 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a2911fd-931a-4d6b-9ca9-b0353c243ade admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:04:55.126590 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 870/1750] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 26 11:04:54 2022] PUT /identity/v3/projects/2c5c7ed5385b4a2dafe1b0f3c4601f64/users/153feb43d9f346ed896b097bd4132526/roles/1f97e4602e2e4d6498913a81f9ae8d6a => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 26 11:04:55.127705 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2911fd-931a-4d6b-9ca9-b0353c243ade admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:04:55.134559 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-3a2911fd-931a-4d6b-9ca9-b0353c243ade admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:04:55.204407 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 11:04:55.232956 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-41f741f2-4264-4eca-901c-0dd2d84c680a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1268423c21a043d38b2400a1ee2deec8', 'request_id': 'req-41f741f2-4264-4eca-901c-0dd2d84c680a'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:04:55.262619 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-41f741f2-4264-4eca-901c-0dd2d84c680a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1268423c21a043d38b2400a1ee2deec8', 'request_id': 'req-41f741f2-4264-4eca-901c-0dd2d84c680a'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:04:55.297603 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-41f741f2-4264-4eca-901c-0dd2d84c680a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1268423c21a043d38b2400a1ee2deec8', 'request_id': 'req-41f741f2-4264-4eca-901c-0dd2d84c680a'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:04:55.353838 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-41f741f2-4264-4eca-901c-0dd2d84c680a admin admin] The token cache is being invalidate because project 1268423c21a043d38b2400a1ee2deec8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81424) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 11:04:55.358277 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-41f741f2-4264-4eca-901c-0dd2d84c680a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:04:55.361695 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-41f741f2-4264-4eca-901c-0dd2d84c680a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:04:55.385758 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 881/1751] 10.222.0.40 () {58 vars in 1233 bytes} [Fri Aug 26 11:04:54 2022] DELETE /identity/v3/projects/1268423c21a043d38b2400a1ee2deec8 => generated 0 bytes in 445 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 11:04:55.388193 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f741f2-4264-4eca-901c-0dd2d84c680a admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:04:55.391271 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-41f741f2-4264-4eca-901c-0dd2d84c680a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:04:55.451044 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2911fd-931a-4d6b-9ca9-b0353c243ade admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:04:55.562415 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f741f2-4264-4eca-901c-0dd2d84c680a admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:04:55.573472 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14a1292-ba60-4f0f-80fe-8a00fd0ab29c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:04:55.575214 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14a1292-ba60-4f0f-80fe-8a00fd0ab29c admin admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:04:55.575890 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14a1292-ba60-4f0f-80fe-8a00fd0ab29c admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:04:55.578154 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14a1292-ba60-4f0f-80fe-8a00fd0ab29c admin admin] PATH_INFO: `/v3/roles` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:04:55.579563 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c14a1292-ba60-4f0f-80fe-8a00fd0ab29c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:04:55.581415 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c14a1292-ba60-4f0f-80fe-8a00fd0ab29c admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:04:55.602881 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 882/1752] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 26 11:04:55 2022] GET /identity/v3/roles => generated 1420 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 11:04:55.627153 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e58ab11-4cc7-417b-8132-d7daf107ff79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:04:55.628028 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f02151e-e425-44a6-93d7-4e75243d102a None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:04:55.630123 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e58ab11-4cc7-417b-8132-d7daf107ff79 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:04:55.630510 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e58ab11-4cc7-417b-8132-d7daf107ff79 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:04:55.630801 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f02151e-e425-44a6-93d7-4e75243d102a None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:04:55.630976 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e58ab11-4cc7-417b-8132-d7daf107ff79 admin admin] PATH_INFO: `/v3/projects/dd36393983104344bda41470fe7358f3` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:04:55.631509 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f02151e-e425-44a6-93d7-4e75243d102a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:04:55.635025 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e58ab11-4cc7-417b-8132-d7daf107ff79 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dd36393983104344bda41470fe7358f3)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:04:55.637949 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e58ab11-4cc7-417b-8132-d7daf107ff79 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:04:55.774973 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-9f02151e-e425-44a6-93d7-4e75243d102a None None] MFA Rules not processed for user `ca155a48e2024c32af3fd2e19e5fac11`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 11:04:55.779836 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-9f02151e-e425-44a6-93d7-4e75243d102a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:04:55.810543 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-1e58ab11-4cc7-417b-8132-d7daf107ff79 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dd36393983104344bda41470fe7358f3', 'request_id': 'req-1e58ab11-4cc7-417b-8132-d7daf107ff79'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:04:55.834072 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-1e58ab11-4cc7-417b-8132-d7daf107ff79 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dd36393983104344bda41470fe7358f3', 'request_id': 'req-1e58ab11-4cc7-417b-8132-d7daf107ff79'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:04:55.846825 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 11:04:55.865680 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-1e58ab11-4cc7-417b-8132-d7daf107ff79 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dd36393983104344bda41470fe7358f3', 'request_id': 'req-1e58ab11-4cc7-417b-8132-d7daf107ff79'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:04:55.913820 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-1e58ab11-4cc7-417b-8132-d7daf107ff79 admin admin] The token cache is being invalidate because project dd36393983104344bda41470fe7358f3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81423) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 11:04:55.914254 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-1e58ab11-4cc7-417b-8132-d7daf107ff79 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:04:55.918752 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-1e58ab11-4cc7-417b-8132-d7daf107ff79 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:04:55.940352 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 871/1753] 10.222.0.40 () {58 vars in 1233 bytes} [Fri Aug 26 11:04:55 2022] DELETE /identity/v3/projects/dd36393983104344bda41470fe7358f3 => generated 0 bytes in 809 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 11:04:55.940352 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e58ab11-4cc7-417b-8132-d7daf107ff79 admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:04:55.953541 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-1e58ab11-4cc7-417b-8132-d7daf107ff79 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:04:56.061760 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 883/1754] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 11:04:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 448 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 11:04:56.070512 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d141445a-8724-445a-b2b6-1b84ae7dbd15 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:04:56.071818 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d141445a-8724-445a-b2b6-1b84ae7dbd15 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:04:56.074197 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d141445a-8724-445a-b2b6-1b84ae7dbd15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:04:56.138970 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-d141445a-8724-445a-b2b6-1b84ae7dbd15 None None] MFA Rules not processed for user `ca155a48e2024c32af3fd2e19e5fac11`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 11:04:56.140518 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-d141445a-8724-445a-b2b6-1b84ae7dbd15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:04:56.178711 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 884/1755] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 11:04:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 11:04:56.181333 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0a421e-6849-49e0-94b8-0320b3b2a81e None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:04:56.182635 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0a421e-6849-49e0-94b8-0320b3b2a81e None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:04:56.183219 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0a421e-6849-49e0-94b8-0320b3b2a81e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:04:56.221559 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 11:04:56.246541 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-7a0a421e-6849-49e0-94b8-0320b3b2a81e None None] MFA Rules not processed for user `ca155a48e2024c32af3fd2e19e5fac11`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 11:04:56.250200 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-7a0a421e-6849-49e0-94b8-0320b3b2a81e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:04:56.280997 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 885/1756] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 11:04:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 11:04:56.284261 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ff6309-f8c0-4bbf-83a6-262e4c636cd7 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:04:56.284781 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ff6309-f8c0-4bbf-83a6-262e4c636cd7 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:04:56.285385 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ff6309-f8c0-4bbf-83a6-262e4c636cd7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:04:56.315817 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e58ab11-4cc7-417b-8132-d7daf107ff79 admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:04:56.325767 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-99ff6309-f8c0-4bbf-83a6-262e4c636cd7 None None] MFA Rules not processed for user `ca155a48e2024c32af3fd2e19e5fac11`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 11:04:56.327963 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-99ff6309-f8c0-4bbf-83a6-262e4c636cd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:04:56.330441 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8455dd2-0956-4259-83d8-fdbc1b0ba8bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:04:56.333324 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8455dd2-0956-4259-83d8-fdbc1b0ba8bc admin admin] REQUEST_METHOD: `PUT` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:04:56.333862 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8455dd2-0956-4259-83d8-fdbc1b0ba8bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:04:56.336327 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8455dd2-0956-4259-83d8-fdbc1b0ba8bc admin admin] PATH_INFO: `/v3/projects/2c5c7ed5385b4a2dafe1b0f3c4601f64/users/153feb43d9f346ed896b097bd4132526/roles/1f97e4602e2e4d6498913a81f9ae8d6a` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:04:56.340909 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 886/1757] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 11:04:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 59 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 11:04:56.342130 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ff6309-f8c0-4bbf-83a6-262e4c636cd7 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:04:56.344682 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8455dd2-0956-4259-83d8-fdbc1b0ba8bc admin admin] RBAC: Authorizing `identity:create_grant(project_id=2c5c7ed5385b4a2dafe1b0f3c4601f64, user_id=153feb43d9f346ed896b097bd4132526, role_id=1f97e4602e2e4d6498913a81f9ae8d6a)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:04:56.358171 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8455dd2-0956-4259-83d8-fdbc1b0ba8bc admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:04:56.358600 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ff6309-f8c0-4bbf-83a6-262e4c636cd7 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:04:56.369024 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6a9356-89c9-44c3-b8eb-8e4bc1e84952 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:04:56.370147 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6a9356-89c9-44c3-b8eb-8e4bc1e84952 admin admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:04:56.370707 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6a9356-89c9-44c3-b8eb-8e4bc1e84952 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:04:56.371103 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6a9356-89c9-44c3-b8eb-8e4bc1e84952 admin admin] PATH_INFO: `/v3/domains` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:04:56.372065 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f6a9356-89c9-44c3-b8eb-8e4bc1e84952 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 11:04:56.372979 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f6a9356-89c9-44c3-b8eb-8e4bc1e84952 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:04:56.374985 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f6a9356-89c9-44c3-b8eb-8e4bc1e84952 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:04:56.423554 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 872/1758] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 26 11:04:55 2022] PUT /identity/v3/projects/2c5c7ed5385b4a2dafe1b0f3c4601f64/users/153feb43d9f346ed896b097bd4132526/roles/1f97e4602e2e4d6498913a81f9ae8d6a => generated 0 bytes in 476 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 11:04:56.423554 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8455dd2-0956-4259-83d8-fdbc1b0ba8bc admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:04:56.458958 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 887/1759] 10.222.0.40 () {58 vars in 1153 bytes} [Fri Aug 26 11:04:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 117 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 11:04:56.462250 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c883ac0c-9eca-47bc-b442-0e78eb80c3b5 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:04:56.465601 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c883ac0c-9eca-47bc-b442-0e78eb80c3b5 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:04:56.466128 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c883ac0c-9eca-47bc-b442-0e78eb80c3b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:04:56.544934 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-c883ac0c-9eca-47bc-b442-0e78eb80c3b5 None None] MFA Rules not processed for user `153feb43d9f346ed896b097bd4132526`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 11:04:56.546580 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-c883ac0c-9eca-47bc-b442-0e78eb80c3b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:04:56.669809 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8455dd2-0956-4259-83d8-fdbc1b0ba8bc admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:04:56.688601 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d652eb-43bf-41b6-891b-e7d877b6e664 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:04:56.689387 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d652eb-43bf-41b6-891b-e7d877b6e664 admin admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:04:56.689899 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d652eb-43bf-41b6-891b-e7d877b6e664 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:04:56.692743 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d652eb-43bf-41b6-891b-e7d877b6e664 admin admin] PATH_INFO: `/v3/domains` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:04:56.693620 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d652eb-43bf-41b6-891b-e7d877b6e664 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 11:04:56.694616 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d652eb-43bf-41b6-891b-e7d877b6e664 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:04:56.703630 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 888/1760] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 11:04:56 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 244 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 11:04:56.705746 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c883ac0c-9eca-47bc-b442-0e78eb80c3b5 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:04:56.710936 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d652eb-43bf-41b6-891b-e7d877b6e664 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:04:56.735811 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c883ac0c-9eca-47bc-b442-0e78eb80c3b5 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:04:56.755118 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef2a8b0-e873-4a80-8712-e29fac87aa6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:04:56.757194 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef2a8b0-e873-4a80-8712-e29fac87aa6e admin admin] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:04:56.757489 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 873/1761] 10.222.0.40 () {58 vars in 1153 bytes} [Fri Aug 26 11:04:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 342 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 11:04:56.757658 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef2a8b0-e873-4a80-8712-e29fac87aa6e admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:04:56.758380 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef2a8b0-e873-4a80-8712-e29fac87aa6e admin admin] PATH_INFO: `/v3/projects` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:04:56.759164 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d652eb-43bf-41b6-891b-e7d877b6e664 admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:04:56.762130 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef2a8b0-e873-4a80-8712-e29fac87aa6e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:04:56.768829 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef2a8b0-e873-4a80-8712-e29fac87aa6e admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:04:56.770015 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-97d652eb-43bf-41b6-891b-e7d877b6e664 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:04:56.911712 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 889/1762] 10.222.0.40 () {60 vars in 1153 bytes} [Fri Aug 26 11:04:56 2022] POST /identity/v3/projects => generated 409 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 11:04:56.913518 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef2a8b0-e873-4a80-8712-e29fac87aa6e admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:04:57.074966 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d652eb-43bf-41b6-891b-e7d877b6e664 admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:04:57.131268 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 11:04:57.168849 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef2a8b0-e873-4a80-8712-e29fac87aa6e admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:04:57.193105 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb0a39b-9eb4-45b8-b419-5211b4e2836c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:04:57.193753 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb0a39b-9eb4-45b8-b419-5211b4e2836c admin admin] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:04:57.194127 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb0a39b-9eb4-45b8-b419-5211b4e2836c admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:04:57.194942 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb0a39b-9eb4-45b8-b419-5211b4e2836c admin admin] PATH_INFO: `/v3/projects` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:04:57.198388 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb0a39b-9eb4-45b8-b419-5211b4e2836c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:04:57.200377 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb0a39b-9eb4-45b8-b419-5211b4e2836c admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:04:57.257642 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebdac9e8-b14b-4489-a706-029db2faf32f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c277338a7f814cc6b01c5419775031bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e30ccf0c1204727b786876c141dbcd2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:04:57.259473 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdac9e8-b14b-4489-a706-029db2faf32f service cinder] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:04:57.259815 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdac9e8-b14b-4489-a706-029db2faf32f service cinder] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:04:57.261470 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdac9e8-b14b-4489-a706-029db2faf32f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:04:57.315481 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 890/1763] 10.222.0.40 () {60 vars in 1153 bytes} [Fri Aug 26 11:04:56 2022] POST /identity/v3/projects => generated 385 bytes in 402 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 11:04:57.320547 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb0a39b-9eb4-45b8-b419-5211b4e2836c admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:04:57.460986 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 11:04:57.523126 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebdac9e8-b14b-4489-a706-029db2faf32f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:04:57.527533 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebdac9e8-b14b-4489-a706-029db2faf32f service cinder] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:04:57.598641 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb0a39b-9eb4-45b8-b419-5211b4e2836c admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:04:57.618503 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 874/1764] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 11:04:56 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 860 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 11:04:57.619486 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebdac9e8-b14b-4489-a706-029db2faf32f service cinder] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:04:57.622504 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b40265-1b81-45f9-b7a2-4e2a9ad44f37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:04:57.623118 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b40265-1b81-45f9-b7a2-4e2a9ad44f37 admin admin] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:04:57.623437 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b40265-1b81-45f9-b7a2-4e2a9ad44f37 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:04:57.623646 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b40265-1b81-45f9-b7a2-4e2a9ad44f37 admin admin] PATH_INFO: `/v3/users` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:04:57.624276 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b40265-1b81-45f9-b7a2-4e2a9ad44f37 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:04:57.635546 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b40265-1b81-45f9-b7a2-4e2a9ad44f37 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:04:57.690452 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebdac9e8-b14b-4489-a706-029db2faf32f service cinder] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:04:57.723493 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946b4a54-76f7-4a1c-9f64-f6630bc2452e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:04:57.724130 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946b4a54-76f7-4a1c-9f64-f6630bc2452e admin admin] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:04:57.724412 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946b4a54-76f7-4a1c-9f64-f6630bc2452e admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:04:57.724686 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946b4a54-76f7-4a1c-9f64-f6630bc2452e admin admin] PATH_INFO: `/v3/users` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:04:57.730215 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 891/1765] 10.222.0.40 () {60 vars in 1144 bytes} [Fri Aug 26 11:04:57 2022] POST /identity/v3/users => generated 305 bytes in 414 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 11:04:57.730944 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-946b4a54-76f7-4a1c-9f64-f6630bc2452e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:04:57.736082 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-946b4a54-76f7-4a1c-9f64-f6630bc2452e admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:04:57.737319 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b40265-1b81-45f9-b7a2-4e2a9ad44f37 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:04:57.771920 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b40265-1b81-45f9-b7a2-4e2a9ad44f37 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:04:57.788780 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022a70a9-0d4f-4bb5-b69b-9ddeec8bead1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:04:57.789490 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022a70a9-0d4f-4bb5-b69b-9ddeec8bead1 admin admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:04:57.789490 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022a70a9-0d4f-4bb5-b69b-9ddeec8bead1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:04:57.789836 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022a70a9-0d4f-4bb5-b69b-9ddeec8bead1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:04:57.790423 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-022a70a9-0d4f-4bb5-b69b-9ddeec8bead1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:04:57.792591 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-022a70a9-0d4f-4bb5-b69b-9ddeec8bead1 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:04:57.799396 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 875/1766] 10.222.0.40 () {60 vars in 1144 bytes} [Fri Aug 26 11:04:57 2022] POST /identity/v3/users => generated 293 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 11:04:57.805546 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946b4a54-76f7-4a1c-9f64-f6630bc2452e admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:04:57.828999 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 892/1767] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 26 11:04:57 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 11:04:57.830197 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022a70a9-0d4f-4bb5-b69b-9ddeec8bead1 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:04:57.878955 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946b4a54-76f7-4a1c-9f64-f6630bc2452e admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:04:57.906457 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b3ade6-33c9-48df-be29-a4eccfc0e848 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:04:57.907360 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b3ade6-33c9-48df-be29-a4eccfc0e848 admin admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:04:57.907787 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b3ade6-33c9-48df-be29-a4eccfc0e848 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:04:57.908205 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b3ade6-33c9-48df-be29-a4eccfc0e848 admin admin] PATH_INFO: `/v3/roles` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:04:57.908997 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b3ade6-33c9-48df-be29-a4eccfc0e848 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:04:57.912952 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b3ade6-33c9-48df-be29-a4eccfc0e848 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:04:57.970016 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 876/1768] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 26 11:04:57 2022] GET /identity/v3/roles => generated 1420 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 11:04:57.970868 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b3ade6-33c9-48df-be29-a4eccfc0e848 admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:04:58.020661 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b3ade6-33c9-48df-be29-a4eccfc0e848 admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:04:58.072547 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056e8077-b0a2-4acb-b4f6-0fe275392b21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:04:58.073104 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056e8077-b0a2-4acb-b4f6-0fe275392b21 admin admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:04:58.073380 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056e8077-b0a2-4acb-b4f6-0fe275392b21 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:04:58.073650 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056e8077-b0a2-4acb-b4f6-0fe275392b21 admin admin] PATH_INFO: `/v3/roles` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:04:58.074326 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-056e8077-b0a2-4acb-b4f6-0fe275392b21 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:04:58.075945 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-056e8077-b0a2-4acb-b4f6-0fe275392b21 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:04:58.112994 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 877/1769] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 26 11:04:57 2022] GET /identity/v3/roles => generated 1420 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 11:04:58.126442 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056e8077-b0a2-4acb-b4f6-0fe275392b21 admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:04:58.138700 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-056e8077-b0a2-4acb-b4f6-0fe275392b21 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:04:58.145715 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022a70a9-0d4f-4bb5-b69b-9ddeec8bead1 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:04:58.171938 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8268cc-2245-451e-99bf-9c153429e170 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c277338a7f814cc6b01c5419775031bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e30ccf0c1204727b786876c141dbcd2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:04:58.172685 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8268cc-2245-451e-99bf-9c153429e170 service cinder] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:04:58.172943 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8268cc-2245-451e-99bf-9c153429e170 service cinder] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:04:58.173218 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8268cc-2245-451e-99bf-9c153429e170 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:04:58.181580 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056e8077-b0a2-4acb-b4f6-0fe275392b21 admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:04:58.197839 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-2c8268cc-2245-451e-99bf-9c153429e170 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:04:58.199543 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b7c371-d103-4ef7-be2f-3aa01795e6d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:04:58.200105 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b7c371-d103-4ef7-be2f-3aa01795e6d4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:04:58.200465 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b7c371-d103-4ef7-be2f-3aa01795e6d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:04:58.200653 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b7c371-d103-4ef7-be2f-3aa01795e6d4 admin admin] PATH_INFO: `/v3/users/8cd0095e694847b49d94bd5e2f80fd57` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:04:58.202517 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b7c371-d103-4ef7-be2f-3aa01795e6d4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8cd0095e694847b49d94bd5e2f80fd57)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:04:58.203951 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b7c371-d103-4ef7-be2f-3aa01795e6d4 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:04:58.348174 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-24b7c371-d103-4ef7-be2f-3aa01795e6d4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8cd0095e694847b49d94bd5e2f80fd57'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:04:58.377475 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-24b7c371-d103-4ef7-be2f-3aa01795e6d4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8cd0095e694847b49d94bd5e2f80fd57'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:04:58.381357 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-24b7c371-d103-4ef7-be2f-3aa01795e6d4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8cd0095e694847b49d94bd5e2f80fd57'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:04:58.404274 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-24b7c371-d103-4ef7-be2f-3aa01795e6d4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8cd0095e694847b49d94bd5e2f80fd57'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:04:58.405677 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-24b7c371-d103-4ef7-be2f-3aa01795e6d4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8cd0095e694847b49d94bd5e2f80fd57'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:04:58.426349 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 878/1770] 10.222.0.40 () {58 vars in 1224 bytes} [Fri Aug 26 11:04:58 2022] DELETE /identity/v3/users/8cd0095e694847b49d94bd5e2f80fd57 => generated 0 bytes in 312 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 11:04:58.427731 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b7c371-d103-4ef7-be2f-3aa01795e6d4 admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:04:58.430288 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-24b7c371-d103-4ef7-be2f-3aa01795e6d4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:04:58.431069 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c8268cc-2245-451e-99bf-9c153429e170 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:04:58.434793 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c8268cc-2245-451e-99bf-9c153429e170 service cinder] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:04:58.484598 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 893/1771] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 11:04:57 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 654 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 11:04:58.486914 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8268cc-2245-451e-99bf-9c153429e170 service cinder] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:04:58.529846 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8268cc-2245-451e-99bf-9c153429e170 service cinder] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:04:58.566766 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 11:04:58.571405 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b7c371-d103-4ef7-be2f-3aa01795e6d4 admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:04:58.591096 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e162886b-8c57-4019-9966-eb4e74b882fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:04:58.592016 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e162886b-8c57-4019-9966-eb4e74b882fe admin admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:04:58.592662 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e162886b-8c57-4019-9966-eb4e74b882fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:04:58.593096 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e162886b-8c57-4019-9966-eb4e74b882fe admin admin] PATH_INFO: `/v3/roles` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:04:58.594653 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e162886b-8c57-4019-9966-eb4e74b882fe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:04:58.596964 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e162886b-8c57-4019-9966-eb4e74b882fe admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:04:58.621079 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 879/1772] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 26 11:04:58 2022] GET /identity/v3/roles => generated 1420 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 11:04:58.626831 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3ff1c9-1bb3-4e9a-91b1-59bb1b0ee44a None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:04:58.629579 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3ff1c9-1bb3-4e9a-91b1-59bb1b0ee44a None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:04:58.630154 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3ff1c9-1bb3-4e9a-91b1-59bb1b0ee44a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:04:58.686574 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-3c3ff1c9-1bb3-4e9a-91b1-59bb1b0ee44a None None] MFA Rules not processed for user `ca155a48e2024c32af3fd2e19e5fac11`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 11:04:58.691336 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-3c3ff1c9-1bb3-4e9a-91b1-59bb1b0ee44a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:04:58.728100 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 880/1773] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 11:04:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 11:04:58.729116 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3ff1c9-1bb3-4e9a-91b1-59bb1b0ee44a None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:04:58.747705 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 11:04:58.762455 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3ff1c9-1bb3-4e9a-91b1-59bb1b0ee44a None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:04:58.783790 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0ec01f-fe45-4575-a54d-d4953c51b559 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:04:58.784678 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0ec01f-fe45-4575-a54d-d4953c51b559 admin admin] REQUEST_METHOD: `PUT` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:04:58.785126 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0ec01f-fe45-4575-a54d-d4953c51b559 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:04:58.785597 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0ec01f-fe45-4575-a54d-d4953c51b559 admin admin] PATH_INFO: `/v3/projects/d196d50f5be14178a60b08107ea0813d/users/b3f12c724b9b492e904278fa6e50386f/roles/23e174ce743b46eabe618c212691da55` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:04:58.828279 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad0ec01f-fe45-4575-a54d-d4953c51b559 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d196d50f5be14178a60b08107ea0813d, user_id=b3f12c724b9b492e904278fa6e50386f, role_id=23e174ce743b46eabe618c212691da55)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:04:58.837341 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad0ec01f-fe45-4575-a54d-d4953c51b559 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:04:58.854880 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd907bb6-95ec-4caa-a732-f393b02cb53b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:04:58.863908 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd907bb6-95ec-4caa-a732-f393b02cb53b admin admin] REQUEST_METHOD: `PUT` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:04:58.864307 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd907bb6-95ec-4caa-a732-f393b02cb53b admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:04:58.870551 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd907bb6-95ec-4caa-a732-f393b02cb53b admin admin] PATH_INFO: `/v3/projects/a0cd6e51377248f58dcde39f5874200a/users/84c54f3982934872b2cf688a5f9507ef/roles/23e174ce743b46eabe618c212691da55` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:04:58.884338 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 881/1774] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 26 11:04:58 2022] PUT /identity/v3/projects/d196d50f5be14178a60b08107ea0813d/users/b3f12c724b9b492e904278fa6e50386f/roles/23e174ce743b46eabe618c212691da55 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 11:04:58.885674 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0ec01f-fe45-4575-a54d-d4953c51b559 admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:04:58.897253 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-ad0ec01f-fe45-4575-a54d-d4953c51b559 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:04:58.983330 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd907bb6-95ec-4caa-a732-f393b02cb53b admin admin] RBAC: Authorizing `identity:create_grant(project_id=a0cd6e51377248f58dcde39f5874200a, user_id=84c54f3982934872b2cf688a5f9507ef, role_id=23e174ce743b46eabe618c212691da55)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:04:59.009255 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd907bb6-95ec-4caa-a732-f393b02cb53b admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:04:59.057725 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0ec01f-fe45-4575-a54d-d4953c51b559 admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:04:59.079482 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 894/1775] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 26 11:04:58 2022] PUT /identity/v3/projects/a0cd6e51377248f58dcde39f5874200a/users/84c54f3982934872b2cf688a5f9507ef/roles/23e174ce743b46eabe618c212691da55 => generated 0 bytes in 585 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 11:04:59.084029 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f1927f-c48e-4712-ad6d-f3dba2ba84b8 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:04:59.084317 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f1927f-c48e-4712-ad6d-f3dba2ba84b8 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:04:59.088102 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f1927f-c48e-4712-ad6d-f3dba2ba84b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:04:59.146503 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3810073b-2946-4491-a9e1-cbce12960c7a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91da2acd2534423091537e1ba068d2b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e30ccf0c1204727b786876c141dbcd2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:04:59.148078 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3810073b-2946-4491-a9e1-cbce12960c7a service neutron] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:04:59.149115 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3810073b-2946-4491-a9e1-cbce12960c7a service neutron] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:04:59.150356 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3810073b-2946-4491-a9e1-cbce12960c7a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:04:59.200107 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-c7f1927f-c48e-4712-ad6d-f3dba2ba84b8 None None] MFA Rules not processed for user `b3f12c724b9b492e904278fa6e50386f`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 11:04:59.204943 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-c7f1927f-c48e-4712-ad6d-f3dba2ba84b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:04:59.301738 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3810073b-2946-4491-a9e1-cbce12960c7a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:04:59.303682 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3810073b-2946-4491-a9e1-cbce12960c7a service neutron] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:04:59.320059 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 882/1776] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 11:04:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 434 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 11:04:59.322115 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6f0292-94bd-45d7-b9fd-f5647eb5bc25 None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:04:59.322405 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6f0292-94bd-45d7-b9fd-f5647eb5bc25 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:04:59.322683 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6f0292-94bd-45d7-b9fd-f5647eb5bc25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:04:59.348694 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-6b6f0292-94bd-45d7-b9fd-f5647eb5bc25 None None] MFA Rules not processed for user `91da2acd2534423091537e1ba068d2b7`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 11:04:59.351307 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-6b6f0292-94bd-45d7-b9fd-f5647eb5bc25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:04:59.361738 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 883/1777] 10.222.0.26 () {58 vars in 1015 bytes} [Fri Aug 26 11:04:59 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 11:04:59.363306 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7c6285-2624-441b-919e-e442913d708b None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:04:59.363597 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7c6285-2624-441b-919e-e442913d708b None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:04:59.363975 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7c6285-2624-441b-919e-e442913d708b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:04:59.408415 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-2f7c6285-2624-441b-919e-e442913d708b None None] MFA Rules not processed for user `84c54f3982934872b2cf688a5f9507ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 11:04:59.409763 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-2f7c6285-2624-441b-919e-e442913d708b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:04:59.438104 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 895/1778] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 11:04:59 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 356 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 11:04:59.452035 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f1927f-c48e-4712-ad6d-f3dba2ba84b8 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:04:59.463557 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 11:04:59.487317 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f1927f-c48e-4712-ad6d-f3dba2ba84b8 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:04:59.526364 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72f59f0-6696-4743-8f7d-4b0972bc71ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:04:59.527009 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72f59f0-6696-4743-8f7d-4b0972bc71ae admin admin] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:04:59.527292 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72f59f0-6696-4743-8f7d-4b0972bc71ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:04:59.527568 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72f59f0-6696-4743-8f7d-4b0972bc71ae admin admin] PATH_INFO: `/v3/projects` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:04:59.531143 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a72f59f0-6696-4743-8f7d-4b0972bc71ae admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:04:59.535771 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 884/1779] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 11:04:59 2022] POST /identity/v3/auth/tokens => generated 3286 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 11:04:59.537102 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7c6285-2624-441b-919e-e442913d708b None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:04:59.548237 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a72f59f0-6696-4743-8f7d-4b0972bc71ae admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:04:59.556693 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7c6285-2624-441b-919e-e442913d708b None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:04:59.563271 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9065ef16-c4aa-4eda-9224-1dee7b9c9292 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91da2acd2534423091537e1ba068d2b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e30ccf0c1204727b786876c141dbcd2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:04:59.564067 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9065ef16-c4aa-4eda-9224-1dee7b9c9292 service neutron] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:04:59.564360 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9065ef16-c4aa-4eda-9224-1dee7b9c9292 service neutron] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:04:59.565073 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9065ef16-c4aa-4eda-9224-1dee7b9c9292 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:04:59.586767 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9065ef16-c4aa-4eda-9224-1dee7b9c9292 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:04:59.589617 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9065ef16-c4aa-4eda-9224-1dee7b9c9292 service neutron] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:04:59.605464 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 885/1780] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 11:04:59 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 69 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 11:04:59.607201 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9065ef16-c4aa-4eda-9224-1dee7b9c9292 service neutron] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:04:59.609203 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-9065ef16-c4aa-4eda-9224-1dee7b9c9292 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:04:59.638105 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9065ef16-c4aa-4eda-9224-1dee7b9c9292 service neutron] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:04:59.642701 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 896/1781] 10.222.0.40 () {60 vars in 1153 bytes} [Fri Aug 26 11:04:59 2022] POST /identity/v3/projects => generated 385 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 11:04:59.651019 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72f59f0-6696-4743-8f7d-4b0972bc71ae admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:04:59.652762 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45bed31-929d-4c2f-b0cb-65410bd847c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:04:59.653612 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45bed31-929d-4c2f-b0cb-65410bd847c6 admin admin] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:04:59.654028 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45bed31-929d-4c2f-b0cb-65410bd847c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:04:59.654568 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45bed31-929d-4c2f-b0cb-65410bd847c6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:04:59.655320 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b45bed31-929d-4c2f-b0cb-65410bd847c6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:04:59.658140 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b45bed31-929d-4c2f-b0cb-65410bd847c6 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:04:59.702015 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 886/1782] 10.222.0.40 () {60 vars in 1153 bytes} [Fri Aug 26 11:04:59 2022] POST /identity/v3/projects => generated 407 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 11:04:59.705615 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45bed31-929d-4c2f-b0cb-65410bd847c6 admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:04:59.861206 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72f59f0-6696-4743-8f7d-4b0972bc71ae admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:04:59.890903 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 11:04:59.933689 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45bed31-929d-4c2f-b0cb-65410bd847c6 admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:04:59.958815 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0dc925a-91a1-4d2d-a3eb-30259936679e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:04:59.959546 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0dc925a-91a1-4d2d-a3eb-30259936679e admin admin] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:04:59.960661 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0dc925a-91a1-4d2d-a3eb-30259936679e admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:04:59.961055 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0dc925a-91a1-4d2d-a3eb-30259936679e admin admin] PATH_INFO: `/v3/users` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:04:59.961770 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0dc925a-91a1-4d2d-a3eb-30259936679e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:04:59.968368 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0dc925a-91a1-4d2d-a3eb-30259936679e admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:05:00.015017 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 887/1783] 10.222.0.40 () {60 vars in 1144 bytes} [Fri Aug 26 11:04:59 2022] POST /identity/v3/users => generated 310 bytes in 310 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 11:05:00.016169 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0dc925a-91a1-4d2d-a3eb-30259936679e admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:05:00.053324 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0dc925a-91a1-4d2d-a3eb-30259936679e admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:05:00.063938 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a55c95-e8db-4382-a8e7-635c848b9b7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:05:00.066705 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a55c95-e8db-4382-a8e7-635c848b9b7b admin admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:05:00.067016 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a55c95-e8db-4382-a8e7-635c848b9b7b admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:05:00.067305 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a55c95-e8db-4382-a8e7-635c848b9b7b admin admin] PATH_INFO: `/v3/roles` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:05:00.069345 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a55c95-e8db-4382-a8e7-635c848b9b7b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:05:00.073448 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a55c95-e8db-4382-a8e7-635c848b9b7b admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:05:00.096688 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 888/1784] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 26 11:05:00 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 11:05:00.102956 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2486f227-5db0-4ae6-9ea9-a7e0e402ee45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:05:00.103867 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a55c95-e8db-4382-a8e7-635c848b9b7b admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:05:00.106167 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2486f227-5db0-4ae6-9ea9-a7e0e402ee45 admin admin] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:05:00.106737 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2486f227-5db0-4ae6-9ea9-a7e0e402ee45 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:05:00.107985 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2486f227-5db0-4ae6-9ea9-a7e0e402ee45 admin admin] PATH_INFO: `/v3/users` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:05:00.112977 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2486f227-5db0-4ae6-9ea9-a7e0e402ee45 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:05:00.119139 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2486f227-5db0-4ae6-9ea9-a7e0e402ee45 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:05:00.125483 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a55c95-e8db-4382-a8e7-635c848b9b7b admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:05:00.134765 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae98d23f-b891-47ea-950b-e16c7b0c3c8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:05:00.135453 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae98d23f-b891-47ea-950b-e16c7b0c3c8a admin admin] REQUEST_METHOD: `PUT` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:05:00.135814 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae98d23f-b891-47ea-950b-e16c7b0c3c8a admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:05:00.136097 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae98d23f-b891-47ea-950b-e16c7b0c3c8a admin admin] PATH_INFO: `/v3/projects/b9e12223e2714aa6b4bad666b07904af/users/426c0f39ef3b473d8c1c881fd1babe2c/roles/1f97e4602e2e4d6498913a81f9ae8d6a` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:05:00.174134 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae98d23f-b891-47ea-950b-e16c7b0c3c8a admin admin] RBAC: Authorizing `identity:create_grant(project_id=b9e12223e2714aa6b4bad666b07904af, user_id=426c0f39ef3b473d8c1c881fd1babe2c, role_id=1f97e4602e2e4d6498913a81f9ae8d6a)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:05:00.179215 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae98d23f-b891-47ea-950b-e16c7b0c3c8a admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:05:00.201724 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 897/1785] 10.222.0.40 () {60 vars in 1144 bytes} [Fri Aug 26 11:04:59 2022] POST /identity/v3/users => generated 293 bytes in 555 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 26 11:05:00.208709 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2486f227-5db0-4ae6-9ea9-a7e0e402ee45 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:05:00.215702 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 889/1786] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 26 11:05:00 2022] PUT /identity/v3/projects/b9e12223e2714aa6b4bad666b07904af/users/426c0f39ef3b473d8c1c881fd1babe2c/roles/1f97e4602e2e4d6498913a81f9ae8d6a => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 11:05:00.219644 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae98d23f-b891-47ea-950b-e16c7b0c3c8a admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:05:00.250015 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2486f227-5db0-4ae6-9ea9-a7e0e402ee45 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:05:00.309824 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 11:05:00.338092 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae98d23f-b891-47ea-950b-e16c7b0c3c8a admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:05:00.346378 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5062a814-b876-4455-ad11-f3d6732e63a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:05:00.347928 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5062a814-b876-4455-ad11-f3d6732e63a3 admin admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:05:00.348482 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5062a814-b876-4455-ad11-f3d6732e63a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:05:00.349448 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5062a814-b876-4455-ad11-f3d6732e63a3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:05:00.351573 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5062a814-b876-4455-ad11-f3d6732e63a3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:05:00.353466 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5062a814-b876-4455-ad11-f3d6732e63a3 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:05:00.399743 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 890/1787] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 26 11:05:00 2022] GET /identity/v3/roles => generated 1420 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 11:05:00.401114 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5062a814-b876-4455-ad11-f3d6732e63a3 admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:05:00.442462 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ff53a3-c7fb-4cea-9cb1-98269ba6d549 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:05:00.443105 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ff53a3-c7fb-4cea-9cb1-98269ba6d549 admin admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:05:00.443381 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ff53a3-c7fb-4cea-9cb1-98269ba6d549 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:05:00.443654 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ff53a3-c7fb-4cea-9cb1-98269ba6d549 admin admin] PATH_INFO: `/v3/roles` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:05:00.450615 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7ff53a3-c7fb-4cea-9cb1-98269ba6d549 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:05:00.450937 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5062a814-b876-4455-ad11-f3d6732e63a3 admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:05:00.455996 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7ff53a3-c7fb-4cea-9cb1-98269ba6d549 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:05:00.485874 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc196ac-d8ae-4778-ba77-ff40dfba768f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:05:00.486262 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc196ac-d8ae-4778-ba77-ff40dfba768f admin admin] REQUEST_METHOD: `PUT` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:05:00.488652 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc196ac-d8ae-4778-ba77-ff40dfba768f admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:05:00.488652 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc196ac-d8ae-4778-ba77-ff40dfba768f admin admin] PATH_INFO: `/v3/projects/b9e12223e2714aa6b4bad666b07904af/users/426c0f39ef3b473d8c1c881fd1babe2c/roles/1f97e4602e2e4d6498913a81f9ae8d6a` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:05:00.495524 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc196ac-d8ae-4778-ba77-ff40dfba768f admin admin] RBAC: Authorizing `identity:create_grant(project_id=b9e12223e2714aa6b4bad666b07904af, user_id=426c0f39ef3b473d8c1c881fd1babe2c, role_id=1f97e4602e2e4d6498913a81f9ae8d6a)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:05:00.502585 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 898/1788] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 26 11:05:00 2022] GET /identity/v3/roles => generated 1420 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 11:05:00.503821 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ff53a3-c7fb-4cea-9cb1-98269ba6d549 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:05:00.505810 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc196ac-d8ae-4778-ba77-ff40dfba768f admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:05:00.548746 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 891/1789] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 26 11:05:00 2022] PUT /identity/v3/projects/b9e12223e2714aa6b4bad666b07904af/users/426c0f39ef3b473d8c1c881fd1babe2c/roles/1f97e4602e2e4d6498913a81f9ae8d6a => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 11:05:00.559433 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1fcbbb-bf13-4f36-94ea-8c8c0c42041e None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:05:00.563971 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ff53a3-c7fb-4cea-9cb1-98269ba6d549 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:05:00.567532 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1fcbbb-bf13-4f36-94ea-8c8c0c42041e None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:05:00.568017 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1fcbbb-bf13-4f36-94ea-8c8c0c42041e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:05:00.674880 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-2a1fcbbb-bf13-4f36-94ea-8c8c0c42041e None None] MFA Rules not processed for user `426c0f39ef3b473d8c1c881fd1babe2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 11:05:00.679598 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-2a1fcbbb-bf13-4f36-94ea-8c8c0c42041e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:05:00.826817 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8bdfbd-fcf5-4149-b7f8-6ac7dca248ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:05:00.827446 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8bdfbd-fcf5-4149-b7f8-6ac7dca248ab admin admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:05:00.827724 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8bdfbd-fcf5-4149-b7f8-6ac7dca248ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:05:00.831320 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8bdfbd-fcf5-4149-b7f8-6ac7dca248ab admin admin] PATH_INFO: `/v3/roles` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:05:00.832076 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b8bdfbd-fcf5-4149-b7f8-6ac7dca248ab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:05:00.833707 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b8bdfbd-fcf5-4149-b7f8-6ac7dca248ab admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:05:00.900937 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 899/1790] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 26 11:05:00 2022] GET /identity/v3/roles => generated 1420 bytes in 397 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 11:05:00.901905 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8bdfbd-fcf5-4149-b7f8-6ac7dca248ab admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:05:00.911860 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 892/1791] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 11:05:00 2022] POST /identity/v3/auth/tokens => generated 3351 bytes in 352 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 11:05:00.927204 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1fcbbb-bf13-4f36-94ea-8c8c0c42041e None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:05:00.929688 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-2a1fcbbb-bf13-4f36-94ea-8c8c0c42041e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:05:00.965713 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8bdfbd-fcf5-4149-b7f8-6ac7dca248ab admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:05:00.989553 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf8a7fa-f196-4947-8593-cbdf05cf8dca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:05:00.992359 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf8a7fa-f196-4947-8593-cbdf05cf8dca admin admin] REQUEST_METHOD: `PUT` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:05:00.996925 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf8a7fa-f196-4947-8593-cbdf05cf8dca admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:05:00.997294 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf8a7fa-f196-4947-8593-cbdf05cf8dca admin admin] PATH_INFO: `/v3/projects/375a02448ae84e8f858713e6ac2c5dd6/users/c62dcaac2b714cbd9a8eb5ccfe93128e/roles/23e174ce743b46eabe618c212691da55` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:05:01.075088 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf8a7fa-f196-4947-8593-cbdf05cf8dca admin admin] RBAC: Authorizing `identity:create_grant(project_id=375a02448ae84e8f858713e6ac2c5dd6, user_id=c62dcaac2b714cbd9a8eb5ccfe93128e, role_id=23e174ce743b46eabe618c212691da55)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:05:01.092210 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1fcbbb-bf13-4f36-94ea-8c8c0c42041e None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:05:01.102801 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc35405-9fa3-4ed2-be5f-3e2dcc9ecaa0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c277338a7f814cc6b01c5419775031bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e30ccf0c1204727b786876c141dbcd2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:05:01.103868 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc35405-9fa3-4ed2-be5f-3e2dcc9ecaa0 service cinder] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:05:01.104162 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc35405-9fa3-4ed2-be5f-3e2dcc9ecaa0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:05:01.105635 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc35405-9fa3-4ed2-be5f-3e2dcc9ecaa0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:05:01.112168 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf8a7fa-f196-4947-8593-cbdf05cf8dca admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:05:01.148062 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc35405-9fa3-4ed2-be5f-3e2dcc9ecaa0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:05:01.157569 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc35405-9fa3-4ed2-be5f-3e2dcc9ecaa0 service cinder] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:05:01.176816 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 900/1792] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 26 11:05:00 2022] PUT /identity/v3/projects/375a02448ae84e8f858713e6ac2c5dd6/users/c62dcaac2b714cbd9a8eb5ccfe93128e/roles/23e174ce743b46eabe618c212691da55 => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 11:05:01.189720 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965b6059-918e-4dd0-8ea0-3bcac5985250 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:05:01.190039 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965b6059-918e-4dd0-8ea0-3bcac5985250 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:05:01.190313 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965b6059-918e-4dd0-8ea0-3bcac5985250 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:05:01.231248 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 893/1793] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 11:05:00 2022] GET /identity/v3/auth/tokens => generated 3351 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 11:05:01.269177 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 11:05:01.342065 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-965b6059-918e-4dd0-8ea0-3bcac5985250 None None] MFA Rules not processed for user `c62dcaac2b714cbd9a8eb5ccfe93128e`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 11:05:01.353550 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-965b6059-918e-4dd0-8ea0-3bcac5985250 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:05:01.629629 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 901/1794] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 11:05:01 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 441 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 11:05:01.635182 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc35405-9fa3-4ed2-be5f-3e2dcc9ecaa0 service cinder] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:05:01.651306 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc35405-9fa3-4ed2-be5f-3e2dcc9ecaa0 service cinder] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:05:01.715892 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965b6059-918e-4dd0-8ea0-3bcac5985250 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:05:01.895402 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b470fb65-e835-4ae1-a440-effda6aff6fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:05:01.896084 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b470fb65-e835-4ae1-a440-effda6aff6fb admin admin] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:05:01.896369 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b470fb65-e835-4ae1-a440-effda6aff6fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:05:01.896649 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b470fb65-e835-4ae1-a440-effda6aff6fb admin admin] PATH_INFO: `/v3/projects` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:05:01.897427 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b470fb65-e835-4ae1-a440-effda6aff6fb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:05:01.900926 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b470fb65-e835-4ae1-a440-effda6aff6fb admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:05:01.918146 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965b6059-918e-4dd0-8ea0-3bcac5985250 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:05:01.923189 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 11:05:01.930497 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664f660a-5996-4e6b-824c-dc4aeffcf294 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c277338a7f814cc6b01c5419775031bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e30ccf0c1204727b786876c141dbcd2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:05:01.931212 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664f660a-5996-4e6b-824c-dc4aeffcf294 service cinder] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:05:01.931491 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664f660a-5996-4e6b-824c-dc4aeffcf294 service cinder] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:05:01.931761 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664f660a-5996-4e6b-824c-dc4aeffcf294 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:05:01.988185 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 894/1795] 10.222.0.40 () {60 vars in 1153 bytes} [Fri Aug 26 11:05:01 2022] POST /identity/v3/projects => generated 385 bytes in 354 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 11:05:01.993438 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b470fb65-e835-4ae1-a440-effda6aff6fb admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:05:02.145106 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-664f660a-5996-4e6b-824c-dc4aeffcf294 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:05:02.146719 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-664f660a-5996-4e6b-824c-dc4aeffcf294 service cinder] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:05:02.167575 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 902/1796] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 11:05:01 2022] GET /identity/v3/auth/tokens => generated 3286 bytes in 452 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 26 11:05:02.202244 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b470fb65-e835-4ae1-a440-effda6aff6fb admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:05:02.222707 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d411046-dcf4-4585-9006-935aaeaa21d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:05:02.231360 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d411046-dcf4-4585-9006-935aaeaa21d0 admin admin] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:05:02.231970 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d411046-dcf4-4585-9006-935aaeaa21d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:05:02.232420 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d411046-dcf4-4585-9006-935aaeaa21d0 admin admin] PATH_INFO: `/v3/users` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:05:02.234244 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d411046-dcf4-4585-9006-935aaeaa21d0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:05:02.242440 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d411046-dcf4-4585-9006-935aaeaa21d0 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:05:02.314597 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 895/1797] 10.222.0.40 () {60 vars in 1144 bytes} [Fri Aug 26 11:05:01 2022] POST /identity/v3/users => generated 299 bytes in 322 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 11:05:02.318749 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664f660a-5996-4e6b-824c-dc4aeffcf294 service cinder] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:05:02.322289 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6330784d-2d9e-4f0a-aebf-7c48fff3ffff None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:05:02.322872 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6330784d-2d9e-4f0a-aebf-7c48fff3ffff None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:05:02.323586 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6330784d-2d9e-4f0a-aebf-7c48fff3ffff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:05:02.339136 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664f660a-5996-4e6b-824c-dc4aeffcf294 service cinder] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:05:02.355632 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11217499-9f39-41c0-bf8b-05dffc5cadd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:05:02.357533 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11217499-9f39-41c0-bf8b-05dffc5cadd7 admin admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:05:02.357995 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11217499-9f39-41c0-bf8b-05dffc5cadd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:05:02.359558 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11217499-9f39-41c0-bf8b-05dffc5cadd7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:05:02.363259 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11217499-9f39-41c0-bf8b-05dffc5cadd7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:05:02.368878 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11217499-9f39-41c0-bf8b-05dffc5cadd7 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:05:02.393642 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-6330784d-2d9e-4f0a-aebf-7c48fff3ffff None None] MFA Rules not processed for user `aedac3f62dd94f87b968f6c71d68a0a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 11:05:02.395749 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-6330784d-2d9e-4f0a-aebf-7c48fff3ffff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:05:02.398813 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 903/1798] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 26 11:05:02 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 11:05:02.403576 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11217499-9f39-41c0-bf8b-05dffc5cadd7 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:05:02.434005 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11217499-9f39-41c0-bf8b-05dffc5cadd7 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:05:02.453421 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1bf773-aac2-4f15-8d32-2f7cf0742951 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:05:02.454983 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1bf773-aac2-4f15-8d32-2f7cf0742951 admin admin] REQUEST_METHOD: `PUT` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:05:02.456991 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1bf773-aac2-4f15-8d32-2f7cf0742951 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:05:02.457309 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1bf773-aac2-4f15-8d32-2f7cf0742951 admin admin] PATH_INFO: `/v3/projects/82f2d67519ee491386963c3b3969391e/users/67ba6b1e818e49e0b0e2aaf18ba0ebf6/roles/1f97e4602e2e4d6498913a81f9ae8d6a` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:05:02.531027 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b1bf773-aac2-4f15-8d32-2f7cf0742951 admin admin] RBAC: Authorizing `identity:create_grant(project_id=82f2d67519ee491386963c3b3969391e, user_id=67ba6b1e818e49e0b0e2aaf18ba0ebf6, role_id=1f97e4602e2e4d6498913a81f9ae8d6a)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:05:02.548958 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b1bf773-aac2-4f15-8d32-2f7cf0742951 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:05:02.567177 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 896/1799] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 11:05:02 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 246 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 11:05:02.601080 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 904/1800] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 26 11:05:02 2022] PUT /identity/v3/projects/82f2d67519ee491386963c3b3969391e/users/67ba6b1e818e49e0b0e2aaf18ba0ebf6/roles/1f97e4602e2e4d6498913a81f9ae8d6a => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 11:05:02.606049 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6330784d-2d9e-4f0a-aebf-7c48fff3ffff None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:05:02.626692 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6330784d-2d9e-4f0a-aebf-7c48fff3ffff None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:05:02.772108 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797c576f-6235-4fc3-b4c6-dc015d6b07c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:05:02.779197 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797c576f-6235-4fc3-b4c6-dc015d6b07c0 admin admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:05:02.779879 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797c576f-6235-4fc3-b4c6-dc015d6b07c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:05:02.780370 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797c576f-6235-4fc3-b4c6-dc015d6b07c0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:05:02.782093 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-797c576f-6235-4fc3-b4c6-dc015d6b07c0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:05:02.785018 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-797c576f-6235-4fc3-b4c6-dc015d6b07c0 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:05:02.814161 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 897/1801] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 26 11:05:02 2022] GET /identity/v3/roles => generated 1420 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 11:05:02.817434 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1bf773-aac2-4f15-8d32-2f7cf0742951 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:05:02.842323 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1bf773-aac2-4f15-8d32-2f7cf0742951 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:05:02.861649 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e8d396-e0e6-450b-82fe-f443d62da0c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:05:02.862910 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e8d396-e0e6-450b-82fe-f443d62da0c6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:05:02.863433 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e8d396-e0e6-450b-82fe-f443d62da0c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:05:02.867899 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e8d396-e0e6-450b-82fe-f443d62da0c6 admin admin] PATH_INFO: `/v3/projects/82f2d67519ee491386963c3b3969391e/users/67ba6b1e818e49e0b0e2aaf18ba0ebf6/roles/1f97e4602e2e4d6498913a81f9ae8d6a` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:05:02.876519 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e8d396-e0e6-450b-82fe-f443d62da0c6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=82f2d67519ee491386963c3b3969391e, user_id=67ba6b1e818e49e0b0e2aaf18ba0ebf6, role_id=1f97e4602e2e4d6498913a81f9ae8d6a)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:05:02.891255 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e8d396-e0e6-450b-82fe-f443d62da0c6 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:05:02.929278 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 905/1802] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 26 11:05:02 2022] PUT /identity/v3/projects/82f2d67519ee491386963c3b3969391e/users/67ba6b1e818e49e0b0e2aaf18ba0ebf6/roles/1f97e4602e2e4d6498913a81f9ae8d6a => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 11:05:02.945952 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aded74cf-bd57-49d6-93bb-ece5457ad41a None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:05:02.946626 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aded74cf-bd57-49d6-93bb-ece5457ad41a None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:05:02.947149 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aded74cf-bd57-49d6-93bb-ece5457ad41a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:05:03.085402 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-aded74cf-bd57-49d6-93bb-ece5457ad41a None None] MFA Rules not processed for user `67ba6b1e818e49e0b0e2aaf18ba0ebf6`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 11:05:03.087000 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-aded74cf-bd57-49d6-93bb-ece5457ad41a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:05:03.119489 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 11:05:03.300052 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 898/1803] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 11:05:02 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 355 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 11:05:03.314237 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e8d396-e0e6-450b-82fe-f443d62da0c6 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:05:03.316723 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-02e8d396-e0e6-450b-82fe-f443d62da0c6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:05:03.324096 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 11:05:03.459449 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e8d396-e0e6-450b-82fe-f443d62da0c6 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:05:03.467312 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b164a6-d8a8-469d-9751-1dbdd807010b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c277338a7f814cc6b01c5419775031bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e30ccf0c1204727b786876c141dbcd2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:05:03.468152 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b164a6-d8a8-469d-9751-1dbdd807010b service cinder] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:05:03.468471 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b164a6-d8a8-469d-9751-1dbdd807010b service cinder] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:05:03.468798 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b164a6-d8a8-469d-9751-1dbdd807010b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:05:03.524630 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b164a6-d8a8-469d-9751-1dbdd807010b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:05:03.527881 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b164a6-d8a8-469d-9751-1dbdd807010b service cinder] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:05:03.571624 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 906/1804] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 11:05:03 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 258 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 11:05:04.269228 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aded74cf-bd57-49d6-93bb-ece5457ad41a None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:05:04.270881 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-aded74cf-bd57-49d6-93bb-ece5457ad41a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:05:04.455213 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aded74cf-bd57-49d6-93bb-ece5457ad41a None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:05:04.478786 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a620986-86d0-47e4-b316-330a0d283f5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:05:04.479620 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a620986-86d0-47e4-b316-330a0d283f5e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:05:04.479996 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a620986-86d0-47e4-b316-330a0d283f5e admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:05:04.480275 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a620986-86d0-47e4-b316-330a0d283f5e admin admin] PATH_INFO: `/v3/projects/9f4328d0c54044d3b1cb751de689fdeb` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:05:04.485572 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a620986-86d0-47e4-b316-330a0d283f5e admin admin] RBAC: Authorizing `identity:delete_project(project_id=9f4328d0c54044d3b1cb751de689fdeb)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:05:04.487704 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a620986-86d0-47e4-b316-330a0d283f5e admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:05:04.586905 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-3a620986-86d0-47e4-b316-330a0d283f5e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9f4328d0c54044d3b1cb751de689fdeb', 'request_id': 'req-3a620986-86d0-47e4-b316-330a0d283f5e'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:05:04.610267 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-3a620986-86d0-47e4-b316-330a0d283f5e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9f4328d0c54044d3b1cb751de689fdeb', 'request_id': 'req-3a620986-86d0-47e4-b316-330a0d283f5e'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:05:04.621582 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-3a620986-86d0-47e4-b316-330a0d283f5e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9f4328d0c54044d3b1cb751de689fdeb', 'request_id': 'req-3a620986-86d0-47e4-b316-330a0d283f5e'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:05:04.661510 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-3a620986-86d0-47e4-b316-330a0d283f5e admin admin] The token cache is being invalidate because project 9f4328d0c54044d3b1cb751de689fdeb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81423) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 11:05:04.661812 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-3a620986-86d0-47e4-b316-330a0d283f5e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:05:04.663973 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-3a620986-86d0-47e4-b316-330a0d283f5e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:05:04.669098 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 899/1805] 10.222.0.40 () {58 vars in 1233 bytes} [Fri Aug 26 11:05:04 2022] DELETE /identity/v3/projects/9f4328d0c54044d3b1cb751de689fdeb => generated 0 bytes in 400 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 11:05:04.673106 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b164a6-d8a8-469d-9751-1dbdd807010b service cinder] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:05:04.693886 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b164a6-d8a8-469d-9751-1dbdd807010b service cinder] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:05:04.801989 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8f0e14-8c81-4382-b6b6-8a66a3fde420 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:05:04.802648 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8f0e14-8c81-4382-b6b6-8a66a3fde420 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:05:04.803092 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8f0e14-8c81-4382-b6b6-8a66a3fde420 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:05:04.805009 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8f0e14-8c81-4382-b6b6-8a66a3fde420 admin admin] PATH_INFO: `/v3/users/53988ed352744aabbcc45d19ed15db6c` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:05:04.806477 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef8f0e14-8c81-4382-b6b6-8a66a3fde420 admin admin] RBAC: Authorizing `identity:delete_user(user_id=53988ed352744aabbcc45d19ed15db6c)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:05:04.808572 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef8f0e14-8c81-4382-b6b6-8a66a3fde420 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:05:04.887448 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5c4188-108a-48e2-aadf-c6dd9ac10bb3 None None] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:05:04.887837 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5c4188-108a-48e2-aadf-c6dd9ac10bb3 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:05:04.888409 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5c4188-108a-48e2-aadf-c6dd9ac10bb3 None None] PATH_INFO: `/` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:05:04.890010 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 900/1806] 10.222.0.40 () {54 vars in 921 bytes} [Fri Aug 26 11:05:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 11:05:04.897307 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf417ba-f1a9-4898-9e9c-c3540e0a7f0b None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:05:04.897813 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf417ba-f1a9-4898-9e9c-c3540e0a7f0b None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:05:04.898148 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf417ba-f1a9-4898-9e9c-c3540e0a7f0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:05:04.915887 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-ef8f0e14-8c81-4382-b6b6-8a66a3fde420 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '53988ed352744aabbcc45d19ed15db6c'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:05:04.916809 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-ef8f0e14-8c81-4382-b6b6-8a66a3fde420 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '53988ed352744aabbcc45d19ed15db6c'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:05:04.928241 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-ef8f0e14-8c81-4382-b6b6-8a66a3fde420 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '53988ed352744aabbcc45d19ed15db6c'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:05:04.930933 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-ef8f0e14-8c81-4382-b6b6-8a66a3fde420 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '53988ed352744aabbcc45d19ed15db6c'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:05:04.952204 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-ef8f0e14-8c81-4382-b6b6-8a66a3fde420 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '53988ed352744aabbcc45d19ed15db6c'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:05:04.966206 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-7bf417ba-f1a9-4898-9e9c-c3540e0a7f0b None None] MFA Rules not processed for user `1f4b4a662db049ecacc865ecff10bdb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 11:05:04.966206 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-7bf417ba-f1a9-4898-9e9c-c3540e0a7f0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:05:04.986507 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 907/1807] 10.222.0.40 () {58 vars in 1224 bytes} [Fri Aug 26 11:05:04 2022] DELETE /identity/v3/users/53988ed352744aabbcc45d19ed15db6c => generated 0 bytes in 314 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 26 11:05:04.987517 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8f0e14-8c81-4382-b6b6-8a66a3fde420 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:05:04.990028 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-ef8f0e14-8c81-4382-b6b6-8a66a3fde420 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:05:05.064418 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 11:05:05.068799 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 11:05:05.134694 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8f0e14-8c81-4382-b6b6-8a66a3fde420 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:05:05.138584 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-986a52f6-f5f4-4763-8e79-cee7171f6fa7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c277338a7f814cc6b01c5419775031bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e30ccf0c1204727b786876c141dbcd2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:05:05.139230 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986a52f6-f5f4-4763-8e79-cee7171f6fa7 service cinder] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:05:05.139510 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986a52f6-f5f4-4763-8e79-cee7171f6fa7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:05:05.139787 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986a52f6-f5f4-4763-8e79-cee7171f6fa7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:05:05.148910 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-986a52f6-f5f4-4763-8e79-cee7171f6fa7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:05:05.281078 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-986a52f6-f5f4-4763-8e79-cee7171f6fa7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:05:05.282600 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-986a52f6-f5f4-4763-8e79-cee7171f6fa7 service cinder] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:05:05.299170 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 901/1808] 10.222.0.40 () {58 vars in 1018 bytes} [Fri Aug 26 11:05:04 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 403 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 11:05:05.308871 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 908/1809] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 11:05:04 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 11:05:05.651523 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf417ba-f1a9-4898-9e9c-c3540e0a7f0b None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:05:05.653940 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-7bf417ba-f1a9-4898-9e9c-c3540e0a7f0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:05:05.701710 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf417ba-f1a9-4898-9e9c-c3540e0a7f0b None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:05:05.865778 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2b49fe-46c6-4114-88f6-b69ba520352f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:05:05.867734 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2b49fe-46c6-4114-88f6-b69ba520352f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:05:05.868176 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2b49fe-46c6-4114-88f6-b69ba520352f admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:05:05.868605 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2b49fe-46c6-4114-88f6-b69ba520352f admin admin] PATH_INFO: `/v3/projects/7dce33deb5d04b26b1ea19ffa7a89555` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:05:05.879173 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f2b49fe-46c6-4114-88f6-b69ba520352f admin admin] RBAC: Authorizing `identity:delete_project(project_id=7dce33deb5d04b26b1ea19ffa7a89555)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:05:05.881716 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f2b49fe-46c6-4114-88f6-b69ba520352f admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:05:05.980492 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-2f2b49fe-46c6-4114-88f6-b69ba520352f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7dce33deb5d04b26b1ea19ffa7a89555', 'request_id': 'req-2f2b49fe-46c6-4114-88f6-b69ba520352f'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:05:05.988231 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-2f2b49fe-46c6-4114-88f6-b69ba520352f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7dce33deb5d04b26b1ea19ffa7a89555', 'request_id': 'req-2f2b49fe-46c6-4114-88f6-b69ba520352f'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:05:05.997169 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-2f2b49fe-46c6-4114-88f6-b69ba520352f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7dce33deb5d04b26b1ea19ffa7a89555', 'request_id': 'req-2f2b49fe-46c6-4114-88f6-b69ba520352f'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:05:06.018196 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-2f2b49fe-46c6-4114-88f6-b69ba520352f admin admin] The token cache is being invalidate because project 7dce33deb5d04b26b1ea19ffa7a89555 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81423) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 11:05:06.018902 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-2f2b49fe-46c6-4114-88f6-b69ba520352f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:05:06.020165 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-2f2b49fe-46c6-4114-88f6-b69ba520352f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:05:06.028637 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 902/1810] 10.222.0.40 () {58 vars in 1233 bytes} [Fri Aug 26 11:05:05 2022] DELETE /identity/v3/projects/7dce33deb5d04b26b1ea19ffa7a89555 => generated 0 bytes in 373 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 11:05:06.029745 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3873b70-486f-4c2c-af59-0621f77157bd None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:05:06.030743 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3873b70-486f-4c2c-af59-0621f77157bd None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:05:06.031210 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3873b70-486f-4c2c-af59-0621f77157bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:05:06.068648 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-d3873b70-486f-4c2c-af59-0621f77157bd None None] MFA Rules not processed for user `ca155a48e2024c32af3fd2e19e5fac11`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 11:05:06.071191 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-d3873b70-486f-4c2c-af59-0621f77157bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:05:06.229486 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2b49fe-46c6-4114-88f6-b69ba520352f admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:05:06.233129 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-2f2b49fe-46c6-4114-88f6-b69ba520352f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:05:06.303466 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 909/1811] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 11:05:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 275 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 11:05:06.306969 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff68d49-3635-43c9-a5d6-a50e41630821 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:05:06.308365 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff68d49-3635-43c9-a5d6-a50e41630821 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:05:06.308763 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff68d49-3635-43c9-a5d6-a50e41630821 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:05:06.363247 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-fff68d49-3635-43c9-a5d6-a50e41630821 None None] MFA Rules not processed for user `163e8be86d5640f88893c50e6f3a8af7`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 11:05:06.364344 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-fff68d49-3635-43c9-a5d6-a50e41630821 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:05:06.521904 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 11:05:06.584716 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 910/1812] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 11:05:06 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 281 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 11:05:06.586268 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab97259c-f3ab-41bf-a634-467cbce6276d None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:05:06.586622 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab97259c-f3ab-41bf-a634-467cbce6276d None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:05:06.587445 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab97259c-f3ab-41bf-a634-467cbce6276d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:05:06.596974 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2b49fe-46c6-4114-88f6-b69ba520352f admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:05:06.618847 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd34e3e9-0f78-4783-9b99-eae184324666 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f4b4a662db049ecacc865ecff10bdb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e30ccf0c1204727b786876c141dbcd2', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:05:06.619492 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd34e3e9-0f78-4783-9b99-eae184324666 service nova] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:05:06.619910 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd34e3e9-0f78-4783-9b99-eae184324666 service nova] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:05:06.620188 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd34e3e9-0f78-4783-9b99-eae184324666 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:05:06.631796 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-cd34e3e9-0f78-4783-9b99-eae184324666 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:05:06.636397 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-ab97259c-f3ab-41bf-a634-467cbce6276d None None] MFA Rules not processed for user `ca155a48e2024c32af3fd2e19e5fac11`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 11:05:06.642153 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-ab97259c-f3ab-41bf-a634-467cbce6276d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:05:06.670518 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd34e3e9-0f78-4783-9b99-eae184324666 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:05:06.672341 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd34e3e9-0f78-4783-9b99-eae184324666 service nova] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:05:06.700114 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 911/1813] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 11:05:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 11:05:06.701134 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab97259c-f3ab-41bf-a634-467cbce6276d None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:05:06.762644 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 903/1814] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 11:05:06 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 533 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 11:05:06.775012 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab97259c-f3ab-41bf-a634-467cbce6276d None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:05:06.817745 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e293b4f-5f99-4af3-8098-a713068e59f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:05:06.818409 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e293b4f-5f99-4af3-8098-a713068e59f6 admin admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:05:06.818685 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e293b4f-5f99-4af3-8098-a713068e59f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:05:06.819424 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e293b4f-5f99-4af3-8098-a713068e59f6 admin admin] PATH_INFO: `/v3/domains` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:05:06.820306 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e293b4f-5f99-4af3-8098-a713068e59f6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 11:05:06.820666 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e293b4f-5f99-4af3-8098-a713068e59f6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:05:06.822372 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e293b4f-5f99-4af3-8098-a713068e59f6 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:05:06.868363 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 912/1815] 10.222.0.40 () {58 vars in 1153 bytes} [Fri Aug 26 11:05:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 11:05:06.872688 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd34e3e9-0f78-4783-9b99-eae184324666 service nova] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:05:06.898584 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd34e3e9-0f78-4783-9b99-eae184324666 service nova] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:05:06.912283 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721afd92-148e-4a1a-9588-743de12f5990 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:05:06.914678 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721afd92-148e-4a1a-9588-743de12f5990 admin admin] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:05:06.916144 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721afd92-148e-4a1a-9588-743de12f5990 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:05:06.916144 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721afd92-148e-4a1a-9588-743de12f5990 admin admin] PATH_INFO: `/v3/projects` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:05:06.919266 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-721afd92-148e-4a1a-9588-743de12f5990 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:05:06.925624 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-721afd92-148e-4a1a-9588-743de12f5990 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:05:06.955753 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4812cb59-1316-458a-83cd-3b758b6145e9 None None] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:05:06.956047 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4812cb59-1316-458a-83cd-3b758b6145e9 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:05:06.956458 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4812cb59-1316-458a-83cd-3b758b6145e9 None None] PATH_INFO: `/` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:05:06.959360 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 913/1816] 10.222.0.40 () {54 vars in 921 bytes} [Fri Aug 26 11:05:06 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 11:05:06.982854 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20dcfaa4-8950-4345-b251-8f5c4b4c68f8 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:05:06.984062 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20dcfaa4-8950-4345-b251-8f5c4b4c68f8 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:05:06.984812 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20dcfaa4-8950-4345-b251-8f5c4b4c68f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:05:07.007429 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 11:05:07.061402 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 904/1817] 10.222.0.40 () {60 vars in 1153 bytes} [Fri Aug 26 11:05:06 2022] POST /identity/v3/projects => generated 381 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 11:05:07.062396 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721afd92-148e-4a1a-9588-743de12f5990 admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:05:07.067353 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-20dcfaa4-8950-4345-b251-8f5c4b4c68f8 None None] MFA Rules not processed for user `1f4b4a662db049ecacc865ecff10bdb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 11:05:07.071101 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-20dcfaa4-8950-4345-b251-8f5c4b4c68f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:05:07.175771 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721afd92-148e-4a1a-9588-743de12f5990 admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:05:07.183477 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758628de-01ec-4054-8598-af959064907c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:05:07.190885 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758628de-01ec-4054-8598-af959064907c admin admin] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:05:07.191426 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758628de-01ec-4054-8598-af959064907c admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:05:07.191864 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758628de-01ec-4054-8598-af959064907c admin admin] PATH_INFO: `/v3/users` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:05:07.192814 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-758628de-01ec-4054-8598-af959064907c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:05:07.195996 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-758628de-01ec-4054-8598-af959064907c admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:05:07.254864 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 905/1818] 10.222.0.40 () {60 vars in 1144 bytes} [Fri Aug 26 11:05:07 2022] POST /identity/v3/users => generated 291 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 11:05:07.264152 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758628de-01ec-4054-8598-af959064907c admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:05:07.291830 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 914/1819] 10.222.0.40 () {58 vars in 1018 bytes} [Fri Aug 26 11:05:06 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 311 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 11:05:07.296278 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758628de-01ec-4054-8598-af959064907c admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:05:07.305223 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6e3ef1-ad39-42fa-89b1-edd640eb416b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:05:07.306407 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6e3ef1-ad39-42fa-89b1-edd640eb416b admin admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:05:07.306783 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6e3ef1-ad39-42fa-89b1-edd640eb416b admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:05:07.307096 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6e3ef1-ad39-42fa-89b1-edd640eb416b admin admin] PATH_INFO: `/v3/roles` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:05:07.307902 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d6e3ef1-ad39-42fa-89b1-edd640eb416b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:05:07.310177 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d6e3ef1-ad39-42fa-89b1-edd640eb416b admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:05:07.357606 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 906/1820] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 26 11:05:07 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 11:05:07.360350 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20dcfaa4-8950-4345-b251-8f5c4b4c68f8 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:05:07.404812 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20dcfaa4-8950-4345-b251-8f5c4b4c68f8 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:05:07.411174 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53551564-97d9-40ac-8e56-1dfdf6b43a53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:05:07.413645 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53551564-97d9-40ac-8e56-1dfdf6b43a53 admin admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:05:07.413988 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53551564-97d9-40ac-8e56-1dfdf6b43a53 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:05:07.414639 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53551564-97d9-40ac-8e56-1dfdf6b43a53 admin admin] PATH_INFO: `/v3/roles` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:05:07.417752 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53551564-97d9-40ac-8e56-1dfdf6b43a53 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:05:07.420005 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53551564-97d9-40ac-8e56-1dfdf6b43a53 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:05:07.442639 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6e3ef1-ad39-42fa-89b1-edd640eb416b admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:05:07.462674 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 915/1821] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 26 11:05:07 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 11:05:07.469330 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6e3ef1-ad39-42fa-89b1-edd640eb416b admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:05:07.477663 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36741341-10a3-4dc5-bf60-5359ed175230 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:05:07.488125 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36741341-10a3-4dc5-bf60-5359ed175230 admin admin] REQUEST_METHOD: `PUT` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:05:07.488563 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36741341-10a3-4dc5-bf60-5359ed175230 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:05:07.489606 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36741341-10a3-4dc5-bf60-5359ed175230 admin admin] PATH_INFO: `/v3/projects/0cf8f160e09549c6b1ef326ba6a59e3a/users/171a00aa58cf47938fdcc608feb0e407/roles/23e174ce743b46eabe618c212691da55` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:05:07.564076 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36741341-10a3-4dc5-bf60-5359ed175230 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0cf8f160e09549c6b1ef326ba6a59e3a, user_id=171a00aa58cf47938fdcc608feb0e407, role_id=23e174ce743b46eabe618c212691da55)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:05:07.581765 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36741341-10a3-4dc5-bf60-5359ed175230 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:05:07.626320 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 907/1822] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 26 11:05:07 2022] PUT /identity/v3/projects/0cf8f160e09549c6b1ef326ba6a59e3a/users/171a00aa58cf47938fdcc608feb0e407/roles/23e174ce743b46eabe618c212691da55 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 11:05:07.634408 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d478d31b-c3b7-4ba0-8561-612d9f48c638 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:05:07.634772 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d478d31b-c3b7-4ba0-8561-612d9f48c638 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:05:07.635007 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d478d31b-c3b7-4ba0-8561-612d9f48c638 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:05:07.714121 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-d478d31b-c3b7-4ba0-8561-612d9f48c638 None None] MFA Rules not processed for user `171a00aa58cf47938fdcc608feb0e407`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 11:05:07.716102 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-d478d31b-c3b7-4ba0-8561-612d9f48c638 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:05:07.835134 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36741341-10a3-4dc5-bf60-5359ed175230 admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:05:07.840429 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 916/1823] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 11:05:07 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 11:05:07.979580 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36741341-10a3-4dc5-bf60-5359ed175230 admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:05:07.994162 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a968e847-c173-435b-a87d-31c0ffaf50d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:05:07.995432 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a968e847-c173-435b-a87d-31c0ffaf50d9 admin admin] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:05:07.995973 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a968e847-c173-435b-a87d-31c0ffaf50d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:05:07.996334 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a968e847-c173-435b-a87d-31c0ffaf50d9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:05:07.998272 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a968e847-c173-435b-a87d-31c0ffaf50d9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:05:08.001184 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a968e847-c173-435b-a87d-31c0ffaf50d9 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:05:08.034667 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 11:05:08.087454 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d478d31b-c3b7-4ba0-8561-612d9f48c638 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:05:08.101766 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 908/1824] 10.222.0.40 () {60 vars in 1153 bytes} [Fri Aug 26 11:05:07 2022] POST /identity/v3/projects => generated 383 bytes in 268 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 11:05:08.109687 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d478d31b-c3b7-4ba0-8561-612d9f48c638 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:05:08.326905 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad75f52-da8a-4e08-ae5d-bf704a0eea2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:05:08.327892 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad75f52-da8a-4e08-ae5d-bf704a0eea2e admin admin] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:05:08.327892 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad75f52-da8a-4e08-ae5d-bf704a0eea2e admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:05:08.328096 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad75f52-da8a-4e08-ae5d-bf704a0eea2e admin admin] PATH_INFO: `/v3/users` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:05:08.328697 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad75f52-da8a-4e08-ae5d-bf704a0eea2e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:05:08.339529 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad75f52-da8a-4e08-ae5d-bf704a0eea2e admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:05:08.401378 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 917/1825] 10.222.0.40 () {60 vars in 1144 bytes} [Fri Aug 26 11:05:08 2022] POST /identity/v3/users => generated 298 bytes in 313 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 11:05:08.402920 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a968e847-c173-435b-a87d-31c0ffaf50d9 admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:05:08.423169 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a968e847-c173-435b-a87d-31c0ffaf50d9 admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:05:08.435516 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59831c40-55cd-4dac-8b16-b42f67b300cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:05:08.438381 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59831c40-55cd-4dac-8b16-b42f67b300cb admin admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:05:08.438836 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59831c40-55cd-4dac-8b16-b42f67b300cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:05:08.439869 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59831c40-55cd-4dac-8b16-b42f67b300cb admin admin] PATH_INFO: `/v3/roles` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:05:08.441695 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59831c40-55cd-4dac-8b16-b42f67b300cb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:05:08.447972 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59831c40-55cd-4dac-8b16-b42f67b300cb admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:05:08.482284 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad75f52-da8a-4e08-ae5d-bf704a0eea2e admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:05:08.483289 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 909/1826] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 26 11:05:08 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 11:05:08.494621 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 11:05:08.502939 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad75f52-da8a-4e08-ae5d-bf704a0eea2e admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:05:08.507500 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59831c40-55cd-4dac-8b16-b42f67b300cb admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:05:08.524861 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d8e9c6-c184-4ad8-9c24-7b81c474e3a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:05:08.527813 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d8e9c6-c184-4ad8-9c24-7b81c474e3a1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:05:08.527813 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d8e9c6-c184-4ad8-9c24-7b81c474e3a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:05:08.527813 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d8e9c6-c184-4ad8-9c24-7b81c474e3a1 admin admin] PATH_INFO: `/v3/projects/5046b33846d34ed99a20077546eb6607/users/3a491492b9ac4033be173548eac543e3/roles/1f97e4602e2e4d6498913a81f9ae8d6a` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:05:08.574966 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d8e9c6-c184-4ad8-9c24-7b81c474e3a1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5046b33846d34ed99a20077546eb6607, user_id=3a491492b9ac4033be173548eac543e3, role_id=1f97e4602e2e4d6498913a81f9ae8d6a)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:05:08.582192 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d8e9c6-c184-4ad8-9c24-7b81c474e3a1 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:05:08.630624 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 918/1827] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 26 11:05:08 2022] PUT /identity/v3/projects/5046b33846d34ed99a20077546eb6607/users/3a491492b9ac4033be173548eac543e3/roles/1f97e4602e2e4d6498913a81f9ae8d6a => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 26 11:05:08.635585 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d8e9c6-c184-4ad8-9c24-7b81c474e3a1 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:05:08.695221 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59831c40-55cd-4dac-8b16-b42f67b300cb admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:05:08.811626 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d8e9c6-c184-4ad8-9c24-7b81c474e3a1 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:05:08.820699 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74959843-e5de-45aa-a3eb-8c321a5b15f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:05:08.822331 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74959843-e5de-45aa-a3eb-8c321a5b15f7 admin admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:05:08.822331 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74959843-e5de-45aa-a3eb-8c321a5b15f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:05:08.822524 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74959843-e5de-45aa-a3eb-8c321a5b15f7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:05:08.825942 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74959843-e5de-45aa-a3eb-8c321a5b15f7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:05:08.830144 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74959843-e5de-45aa-a3eb-8c321a5b15f7 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:05:08.854138 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 919/1828] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 26 11:05:08 2022] GET /identity/v3/roles => generated 1420 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 26 11:05:08.858572 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74959843-e5de-45aa-a3eb-8c321a5b15f7 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:05:08.872821 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74959843-e5de-45aa-a3eb-8c321a5b15f7 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:05:08.873287 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f51ddd-7ee1-4b6f-b9b1-0a5f5c8530b4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f4b4a662db049ecacc865ecff10bdb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e30ccf0c1204727b786876c141dbcd2', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:05:08.873916 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f51ddd-7ee1-4b6f-b9b1-0a5f5c8530b4 service nova] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:05:08.874314 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f51ddd-7ee1-4b6f-b9b1-0a5f5c8530b4 service nova] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:05:08.874663 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f51ddd-7ee1-4b6f-b9b1-0a5f5c8530b4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:05:08.879210 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9cbad76-c068-46b5-b41a-7c7f9fa586b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:05:08.879987 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9cbad76-c068-46b5-b41a-7c7f9fa586b7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:05:08.880376 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9cbad76-c068-46b5-b41a-7c7f9fa586b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:05:08.881090 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9cbad76-c068-46b5-b41a-7c7f9fa586b7 admin admin] PATH_INFO: `/v3/projects/5046b33846d34ed99a20077546eb6607/users/3a491492b9ac4033be173548eac543e3/roles/1f97e4602e2e4d6498913a81f9ae8d6a` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:05:08.884045 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9cbad76-c068-46b5-b41a-7c7f9fa586b7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5046b33846d34ed99a20077546eb6607, user_id=3a491492b9ac4033be173548eac543e3, role_id=1f97e4602e2e4d6498913a81f9ae8d6a)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:05:08.886111 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f51ddd-7ee1-4b6f-b9b1-0a5f5c8530b4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:05:08.890051 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9cbad76-c068-46b5-b41a-7c7f9fa586b7 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:05:08.890218 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f51ddd-7ee1-4b6f-b9b1-0a5f5c8530b4 service nova] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:05:08.901773 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 920/1830] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 26 11:05:08 2022] PUT /identity/v3/projects/5046b33846d34ed99a20077546eb6607/users/3a491492b9ac4033be173548eac543e3/roles/1f97e4602e2e4d6498913a81f9ae8d6a => generated 0 bytes in 44 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 26 11:05:08.901912 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 910/1830] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 11:05:08 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 395 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 26 11:05:08.911636 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6a7876-e6ef-476e-80df-e6e28b8934bf None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:05:08.912415 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6a7876-e6ef-476e-80df-e6e28b8934bf None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:05:08.913275 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6a7876-e6ef-476e-80df-e6e28b8934bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:05:08.954621 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-9a6a7876-e6ef-476e-80df-e6e28b8934bf None None] MFA Rules not processed for user `3a491492b9ac4033be173548eac543e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 11:05:08.956352 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-9a6a7876-e6ef-476e-80df-e6e28b8934bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:05:09.081045 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 921/1831] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 11:05:08 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 11:05:09.093555 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f51ddd-7ee1-4b6f-b9b1-0a5f5c8530b4 service nova] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:05:09.095300 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-56f51ddd-7ee1-4b6f-b9b1-0a5f5c8530b4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:05:09.183806 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f51ddd-7ee1-4b6f-b9b1-0a5f5c8530b4 service nova] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:05:09.213700 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 11:05:09.260318 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae72da38-2b07-4f67-aecd-a1829aa4684c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c277338a7f814cc6b01c5419775031bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e30ccf0c1204727b786876c141dbcd2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:05:09.261012 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae72da38-2b07-4f67-aecd-a1829aa4684c service cinder] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:05:09.261363 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae72da38-2b07-4f67-aecd-a1829aa4684c service cinder] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:05:09.261901 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae72da38-2b07-4f67-aecd-a1829aa4684c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:05:09.274644 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae72da38-2b07-4f67-aecd-a1829aa4684c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:05:09.276159 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae72da38-2b07-4f67-aecd-a1829aa4684c service cinder] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:05:09.304336 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 911/1832] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 11:05:09 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 212 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 11:05:09.420253 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6a7876-e6ef-476e-80df-e6e28b8934bf None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:05:09.431781 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6a7876-e6ef-476e-80df-e6e28b8934bf None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:05:09.436574 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f957b5-bba8-410c-9bc9-265ea81de4b3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c277338a7f814cc6b01c5419775031bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e30ccf0c1204727b786876c141dbcd2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:05:09.437374 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f957b5-bba8-410c-9bc9-265ea81de4b3 service cinder] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:05:09.437671 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f957b5-bba8-410c-9bc9-265ea81de4b3 service cinder] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:05:09.438031 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f957b5-bba8-410c-9bc9-265ea81de4b3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:05:09.486371 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 11:05:09.500845 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f957b5-bba8-410c-9bc9-265ea81de4b3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:05:09.502328 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f957b5-bba8-410c-9bc9-265ea81de4b3 service cinder] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:05:09.513636 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 922/1833] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 11:05:09 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 11:05:12.208802 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae72da38-2b07-4f67-aecd-a1829aa4684c service cinder] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:05:12.224206 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae72da38-2b07-4f67-aecd-a1829aa4684c service cinder] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:05:12.228072 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b8631c-0da5-4822-b503-43806aa7fc87 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c277338a7f814cc6b01c5419775031bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e30ccf0c1204727b786876c141dbcd2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:05:12.228708 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b8631c-0da5-4822-b503-43806aa7fc87 service cinder] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:05:12.228988 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b8631c-0da5-4822-b503-43806aa7fc87 service cinder] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:05:12.229258 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b8631c-0da5-4822-b503-43806aa7fc87 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:05:12.231118 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-b7b8631c-0da5-4822-b503-43806aa7fc87 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:05:12.295157 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b8631c-0da5-4822-b503-43806aa7fc87 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:05:12.296613 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b8631c-0da5-4822-b503-43806aa7fc87 service cinder] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:05:12.307522 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 912/1834] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 11:05:12 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 11:05:13.282130 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f957b5-bba8-410c-9bc9-265ea81de4b3 service cinder] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:05:13.374128 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f957b5-bba8-410c-9bc9-265ea81de4b3 service cinder] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:05:13.380148 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19431f69-62df-4400-a896-2dd768a5850c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:05:13.381159 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19431f69-62df-4400-a896-2dd768a5850c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:05:13.381386 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19431f69-62df-4400-a896-2dd768a5850c admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:05:13.381655 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19431f69-62df-4400-a896-2dd768a5850c admin admin] PATH_INFO: `/v3/users/171a00aa58cf47938fdcc608feb0e407` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:05:13.383591 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19431f69-62df-4400-a896-2dd768a5850c admin admin] RBAC: Authorizing `identity:delete_user(user_id=171a00aa58cf47938fdcc608feb0e407)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:05:13.385610 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19431f69-62df-4400-a896-2dd768a5850c admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:05:13.438290 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-19431f69-62df-4400-a896-2dd768a5850c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '171a00aa58cf47938fdcc608feb0e407'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:05:13.439117 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-19431f69-62df-4400-a896-2dd768a5850c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '171a00aa58cf47938fdcc608feb0e407'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:05:13.446570 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-19431f69-62df-4400-a896-2dd768a5850c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '171a00aa58cf47938fdcc608feb0e407'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:05:13.447219 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-19431f69-62df-4400-a896-2dd768a5850c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '171a00aa58cf47938fdcc608feb0e407'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:05:13.459280 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-19431f69-62df-4400-a896-2dd768a5850c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '171a00aa58cf47938fdcc608feb0e407'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:05:13.473341 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 923/1835] 10.222.0.40 () {58 vars in 1224 bytes} [Fri Aug 26 11:05:13 2022] DELETE /identity/v3/users/171a00aa58cf47938fdcc608feb0e407 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 11:05:13.478615 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f72111-61a6-45f8-ae6b-b4542fc34908 None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:05:13.479067 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f72111-61a6-45f8-ae6b-b4542fc34908 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:05:13.479183 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f72111-61a6-45f8-ae6b-b4542fc34908 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:05:13.498292 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-c4f72111-61a6-45f8-ae6b-b4542fc34908 None None] MFA Rules not processed for user `ca155a48e2024c32af3fd2e19e5fac11`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 11:05:13.499511 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-c4f72111-61a6-45f8-ae6b-b4542fc34908 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:05:13.563104 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 913/1836] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 11:05:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 11:05:13.573428 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19431f69-62df-4400-a896-2dd768a5850c admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:05:13.574882 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-19431f69-62df-4400-a896-2dd768a5850c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:05:13.635987 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19431f69-62df-4400-a896-2dd768a5850c admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:05:13.639831 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ffd05a-2f76-48b6-ba41-bd7ee68266e7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91da2acd2534423091537e1ba068d2b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e30ccf0c1204727b786876c141dbcd2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:05:13.640517 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ffd05a-2f76-48b6-ba41-bd7ee68266e7 service neutron] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:05:13.640864 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ffd05a-2f76-48b6-ba41-bd7ee68266e7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:05:13.641264 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ffd05a-2f76-48b6-ba41-bd7ee68266e7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:05:13.652401 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ffd05a-2f76-48b6-ba41-bd7ee68266e7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:05:13.653824 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ffd05a-2f76-48b6-ba41-bd7ee68266e7 service neutron] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:05:13.664928 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 924/1837] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 11:05:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 11:05:14.034563 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f72111-61a6-45f8-ae6b-b4542fc34908 None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:05:14.036601 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-c4f72111-61a6-45f8-ae6b-b4542fc34908 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:05:14.073676 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f72111-61a6-45f8-ae6b-b4542fc34908 None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:05:14.100733 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f9823a-278a-46ef-819f-ac94d6f35548 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:05:14.101067 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f9823a-278a-46ef-819f-ac94d6f35548 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:05:14.101697 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f9823a-278a-46ef-819f-ac94d6f35548 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:05:14.102693 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f9823a-278a-46ef-819f-ac94d6f35548 admin admin] PATH_INFO: `/v3/projects/0cf8f160e09549c6b1ef326ba6a59e3a` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:05:14.103548 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f9823a-278a-46ef-819f-ac94d6f35548 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0cf8f160e09549c6b1ef326ba6a59e3a)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:05:14.106628 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f9823a-278a-46ef-819f-ac94d6f35548 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:05:14.174819 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-a3f9823a-278a-46ef-819f-ac94d6f35548 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0cf8f160e09549c6b1ef326ba6a59e3a', 'request_id': 'req-a3f9823a-278a-46ef-819f-ac94d6f35548'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:05:14.182917 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-a3f9823a-278a-46ef-819f-ac94d6f35548 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0cf8f160e09549c6b1ef326ba6a59e3a', 'request_id': 'req-a3f9823a-278a-46ef-819f-ac94d6f35548'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:05:14.189037 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 11:05:14.191965 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-a3f9823a-278a-46ef-819f-ac94d6f35548 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0cf8f160e09549c6b1ef326ba6a59e3a', 'request_id': 'req-a3f9823a-278a-46ef-819f-ac94d6f35548'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:05:14.207400 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-a3f9823a-278a-46ef-819f-ac94d6f35548 admin admin] The token cache is being invalidate because project 0cf8f160e09549c6b1ef326ba6a59e3a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81423) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 11:05:14.207689 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-a3f9823a-278a-46ef-819f-ac94d6f35548 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:05:14.208339 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-a3f9823a-278a-46ef-819f-ac94d6f35548 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:05:14.211092 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 914/1838] 10.222.0.40 () {58 vars in 1233 bytes} [Fri Aug 26 11:05:14 2022] DELETE /identity/v3/projects/0cf8f160e09549c6b1ef326ba6a59e3a => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 11:05:14.215097 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ffd05a-2f76-48b6-ba41-bd7ee68266e7 service neutron] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:05:14.224905 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ffd05a-2f76-48b6-ba41-bd7ee68266e7 service neutron] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:05:14.233868 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 11:05:14.307875 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc96bb72-9c72-42b1-b085-9729c4bd6df9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:05:14.308730 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc96bb72-9c72-42b1-b085-9729c4bd6df9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:05:14.309163 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc96bb72-9c72-42b1-b085-9729c4bd6df9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:05:14.309451 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc96bb72-9c72-42b1-b085-9729c4bd6df9 admin admin] PATH_INFO: `/v3/users/3a491492b9ac4033be173548eac543e3` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:05:14.310867 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc96bb72-9c72-42b1-b085-9729c4bd6df9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3a491492b9ac4033be173548eac543e3)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:05:14.312810 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc96bb72-9c72-42b1-b085-9729c4bd6df9 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:05:14.381819 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-dc96bb72-9c72-42b1-b085-9729c4bd6df9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3a491492b9ac4033be173548eac543e3'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:05:14.382599 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-dc96bb72-9c72-42b1-b085-9729c4bd6df9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3a491492b9ac4033be173548eac543e3'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:05:14.390342 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-dc96bb72-9c72-42b1-b085-9729c4bd6df9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3a491492b9ac4033be173548eac543e3'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:05:14.391086 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-dc96bb72-9c72-42b1-b085-9729c4bd6df9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3a491492b9ac4033be173548eac543e3'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:05:14.408415 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-dc96bb72-9c72-42b1-b085-9729c4bd6df9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3a491492b9ac4033be173548eac543e3'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:05:14.434117 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 925/1839] 10.222.0.40 () {58 vars in 1224 bytes} [Fri Aug 26 11:05:14 2022] DELETE /identity/v3/users/3a491492b9ac4033be173548eac543e3 => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 11:05:14.758491 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f9823a-278a-46ef-819f-ac94d6f35548 admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:05:14.760258 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-a3f9823a-278a-46ef-819f-ac94d6f35548 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:05:14.775860 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f9823a-278a-46ef-819f-ac94d6f35548 admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:05:14.849730 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411954ab-34ba-430f-a760-713ec221a377 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:05:14.850688 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411954ab-34ba-430f-a760-713ec221a377 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:05:14.851048 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411954ab-34ba-430f-a760-713ec221a377 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:05:14.851327 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411954ab-34ba-430f-a760-713ec221a377 admin admin] PATH_INFO: `/v3/projects/5046b33846d34ed99a20077546eb6607` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:05:14.852462 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-411954ab-34ba-430f-a760-713ec221a377 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5046b33846d34ed99a20077546eb6607)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:05:14.854434 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-411954ab-34ba-430f-a760-713ec221a377 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:05:14.908527 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-411954ab-34ba-430f-a760-713ec221a377 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5046b33846d34ed99a20077546eb6607', 'request_id': 'req-411954ab-34ba-430f-a760-713ec221a377'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:05:14.915225 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-411954ab-34ba-430f-a760-713ec221a377 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5046b33846d34ed99a20077546eb6607', 'request_id': 'req-411954ab-34ba-430f-a760-713ec221a377'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:05:14.921749 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-411954ab-34ba-430f-a760-713ec221a377 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5046b33846d34ed99a20077546eb6607', 'request_id': 'req-411954ab-34ba-430f-a760-713ec221a377'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:05:14.937296 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-411954ab-34ba-430f-a760-713ec221a377 admin admin] The token cache is being invalidate because project 5046b33846d34ed99a20077546eb6607 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81423) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 11:05:14.937583 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-411954ab-34ba-430f-a760-713ec221a377 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:05:14.938229 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-411954ab-34ba-430f-a760-713ec221a377 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:05:14.939744 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 915/1840] 10.222.0.40 () {58 vars in 1233 bytes} [Fri Aug 26 11:05:14 2022] DELETE /identity/v3/projects/5046b33846d34ed99a20077546eb6607 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 11:05:14.945450 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5cdb6b3-b275-4736-a786-e7ad9df12c52 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:05:14.945742 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5cdb6b3-b275-4736-a786-e7ad9df12c52 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:05:14.946076 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5cdb6b3-b275-4736-a786-e7ad9df12c52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:05:14.971421 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-d5cdb6b3-b275-4736-a786-e7ad9df12c52 None None] MFA Rules not processed for user `ca155a48e2024c32af3fd2e19e5fac11`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 11:05:14.972711 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-d5cdb6b3-b275-4736-a786-e7ad9df12c52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:05:15.007849 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411954ab-34ba-430f-a760-713ec221a377 admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:05:15.009339 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-411954ab-34ba-430f-a760-713ec221a377 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:05:15.086023 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 926/1841] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 11:05:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 11:05:15.093321 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d35d962-c90b-4d4d-9629-ba451dc68285 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:05:15.094158 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d35d962-c90b-4d4d-9629-ba451dc68285 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:05:15.094158 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d35d962-c90b-4d4d-9629-ba451dc68285 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:05:15.099165 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411954ab-34ba-430f-a760-713ec221a377 admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:05:15.105426 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5660cb-ea13-47dc-bf29-2f35ce294b97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:05:15.106571 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5660cb-ea13-47dc-bf29-2f35ce294b97 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:05:15.106571 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5660cb-ea13-47dc-bf29-2f35ce294b97 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:05:15.106773 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5660cb-ea13-47dc-bf29-2f35ce294b97 admin admin] PATH_INFO: `/v3/users/b3f12c724b9b492e904278fa6e50386f` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:05:15.107858 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f5660cb-ea13-47dc-bf29-2f35ce294b97 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b3f12c724b9b492e904278fa6e50386f)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:05:15.109780 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f5660cb-ea13-47dc-bf29-2f35ce294b97 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:05:15.118331 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-1d35d962-c90b-4d4d-9629-ba451dc68285 None None] MFA Rules not processed for user `ca155a48e2024c32af3fd2e19e5fac11`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 11:05:15.119666 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-1d35d962-c90b-4d4d-9629-ba451dc68285 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:05:15.130017 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 927/1842] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 11:05:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 11:05:15.134242 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d35d962-c90b-4d4d-9629-ba451dc68285 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:05:15.146547 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d35d962-c90b-4d4d-9629-ba451dc68285 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:05:15.152934 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e16b309-e7e3-4204-a781-c575c36bb941 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:05:15.153875 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e16b309-e7e3-4204-a781-c575c36bb941 admin admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:05:15.154328 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e16b309-e7e3-4204-a781-c575c36bb941 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:05:15.154955 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e16b309-e7e3-4204-a781-c575c36bb941 admin admin] PATH_INFO: `/v3/domains` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:05:15.156056 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e16b309-e7e3-4204-a781-c575c36bb941 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 11:05:15.156609 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e16b309-e7e3-4204-a781-c575c36bb941 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:05:15.159181 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=87,keystone:INSERT=2,keystone:DELETE=11 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 11:05:15.163691 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e16b309-e7e3-4204-a781-c575c36bb941 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:05:15.185495 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 928/1843] 10.222.0.40 () {58 vars in 1153 bytes} [Fri Aug 26 11:05:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 11:05:15.189481 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e16b309-e7e3-4204-a781-c575c36bb941 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:05:15.201051 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-6f5660cb-ea13-47dc-bf29-2f35ce294b97 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b3f12c724b9b492e904278fa6e50386f'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:05:15.214783 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-6f5660cb-ea13-47dc-bf29-2f35ce294b97 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b3f12c724b9b492e904278fa6e50386f'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:05:15.215914 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-6f5660cb-ea13-47dc-bf29-2f35ce294b97 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b3f12c724b9b492e904278fa6e50386f'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:05:15.222040 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e16b309-e7e3-4204-a781-c575c36bb941 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:05:15.229458 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08b961c-76e3-40b9-bc7d-669c2056eaa7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:05:15.230216 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08b961c-76e3-40b9-bc7d-669c2056eaa7 admin admin] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:05:15.230566 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08b961c-76e3-40b9-bc7d-669c2056eaa7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:05:15.231063 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08b961c-76e3-40b9-bc7d-669c2056eaa7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:05:15.231784 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b08b961c-76e3-40b9-bc7d-669c2056eaa7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:05:15.235052 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b08b961c-76e3-40b9-bc7d-669c2056eaa7 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:05:15.242790 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 11:05:15.258721 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-6f5660cb-ea13-47dc-bf29-2f35ce294b97 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b3f12c724b9b492e904278fa6e50386f'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:05:15.259696 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-6f5660cb-ea13-47dc-bf29-2f35ce294b97 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b3f12c724b9b492e904278fa6e50386f'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:05:15.284462 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 916/1844] 10.222.0.40 () {58 vars in 1224 bytes} [Fri Aug 26 11:05:15 2022] DELETE /identity/v3/users/b3f12c724b9b492e904278fa6e50386f => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 11:05:15.286547 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4808d3a3-ed26-4961-974e-3a3d9a924b0b None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:05:15.287059 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4808d3a3-ed26-4961-974e-3a3d9a924b0b None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:05:15.287504 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4808d3a3-ed26-4961-974e-3a3d9a924b0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:05:15.289702 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 929/1845] 10.222.0.40 () {60 vars in 1153 bytes} [Fri Aug 26 11:05:15 2022] POST /identity/v3/projects => generated 389 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 11:05:15.294835 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08b961c-76e3-40b9-bc7d-669c2056eaa7 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:05:15.307444 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-b08b961c-76e3-40b9-bc7d-669c2056eaa7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:05:15.324358 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-4808d3a3-ed26-4961-974e-3a3d9a924b0b None None] MFA Rules not processed for user `ca155a48e2024c32af3fd2e19e5fac11`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 11:05:15.325891 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-4808d3a3-ed26-4961-974e-3a3d9a924b0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:05:15.393753 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08b961c-76e3-40b9-bc7d-669c2056eaa7 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:05:15.400479 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b9dec0-0f68-4d48-a5b8-ed7d89018583 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:05:15.401103 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b9dec0-0f68-4d48-a5b8-ed7d89018583 admin admin] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:05:15.401478 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b9dec0-0f68-4d48-a5b8-ed7d89018583 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:05:15.402632 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b9dec0-0f68-4d48-a5b8-ed7d89018583 admin admin] PATH_INFO: `/v3/users` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:05:15.403275 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b9dec0-0f68-4d48-a5b8-ed7d89018583 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:05:15.406257 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b9dec0-0f68-4d48-a5b8-ed7d89018583 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:05:15.422752 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 917/1846] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 11:05:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 11:05:15.441394 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 930/1847] 10.222.0.40 () {60 vars in 1144 bytes} [Fri Aug 26 11:05:15 2022] POST /identity/v3/users => generated 295 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 11:05:15.445674 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4808d3a3-ed26-4961-974e-3a3d9a924b0b None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:05:15.457773 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4808d3a3-ed26-4961-974e-3a3d9a924b0b None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:05:15.463576 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a618d9-1d01-42b6-bb44-795bdbca952d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:05:15.464254 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a618d9-1d01-42b6-bb44-795bdbca952d admin admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:05:15.464537 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a618d9-1d01-42b6-bb44-795bdbca952d admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:05:15.465072 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a618d9-1d01-42b6-bb44-795bdbca952d admin admin] PATH_INFO: `/v3/roles` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:05:15.465772 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a618d9-1d01-42b6-bb44-795bdbca952d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:05:15.467743 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a618d9-1d01-42b6-bb44-795bdbca952d admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:05:15.471947 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b9dec0-0f68-4d48-a5b8-ed7d89018583 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:05:15.473885 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-29b9dec0-0f68-4d48-a5b8-ed7d89018583 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:05:15.482464 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 918/1848] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 26 11:05:15 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 11:05:15.485562 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a618d9-1d01-42b6-bb44-795bdbca952d admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:05:15.501284 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a618d9-1d01-42b6-bb44-795bdbca952d admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:05:15.507821 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c81bd71-e44f-4c52-b19d-77d48d457027 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:05:15.508761 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c81bd71-e44f-4c52-b19d-77d48d457027 admin admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:05:15.509141 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c81bd71-e44f-4c52-b19d-77d48d457027 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:05:15.509506 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c81bd71-e44f-4c52-b19d-77d48d457027 admin admin] PATH_INFO: `/v3/roles` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:05:15.510330 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c81bd71-e44f-4c52-b19d-77d48d457027 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:05:15.512076 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c81bd71-e44f-4c52-b19d-77d48d457027 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:05:15.525297 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 919/1849] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 26 11:05:15 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 26 11:05:15.541595 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c81bd71-e44f-4c52-b19d-77d48d457027 admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:05:15.556928 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c81bd71-e44f-4c52-b19d-77d48d457027 admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:05:15.563818 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea639a4-e006-4c93-bc83-de572db336c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:05:15.566731 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea639a4-e006-4c93-bc83-de572db336c9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:05:15.566731 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea639a4-e006-4c93-bc83-de572db336c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:05:15.566731 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea639a4-e006-4c93-bc83-de572db336c9 admin admin] PATH_INFO: `/v3/projects/c9cd769db7204342904cf450ba649e27/users/97e0a839a2d541efb7529d19f530abf9/roles/23e174ce743b46eabe618c212691da55` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:05:15.617960 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea639a4-e006-4c93-bc83-de572db336c9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c9cd769db7204342904cf450ba649e27, user_id=97e0a839a2d541efb7529d19f530abf9, role_id=23e174ce743b46eabe618c212691da55)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:05:15.618874 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b9dec0-0f68-4d48-a5b8-ed7d89018583 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:05:15.622829 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea639a4-e006-4c93-bc83-de572db336c9 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:05:15.638417 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5e2caa-c6ad-4e6c-8496-af2c8afe77d2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91da2acd2534423091537e1ba068d2b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e30ccf0c1204727b786876c141dbcd2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:05:15.640089 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5e2caa-c6ad-4e6c-8496-af2c8afe77d2 service neutron] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:05:15.641001 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5e2caa-c6ad-4e6c-8496-af2c8afe77d2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:05:15.641900 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5e2caa-c6ad-4e6c-8496-af2c8afe77d2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:05:15.657796 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 920/1850] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 26 11:05:15 2022] PUT /identity/v3/projects/c9cd769db7204342904cf450ba649e27/users/97e0a839a2d541efb7529d19f530abf9/roles/23e174ce743b46eabe618c212691da55 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 11:05:15.659475 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c6ccb4-0f64-4b22-9e88-7821814f13ce None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:05:15.659674 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c6ccb4-0f64-4b22-9e88-7821814f13ce None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:05:15.660005 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c6ccb4-0f64-4b22-9e88-7821814f13ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:05:15.683547 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b5e2caa-c6ad-4e6c-8496-af2c8afe77d2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:05:15.687113 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b5e2caa-c6ad-4e6c-8496-af2c8afe77d2 service neutron] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:05:15.701665 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-60c6ccb4-0f64-4b22-9e88-7821814f13ce None None] MFA Rules not processed for user `97e0a839a2d541efb7529d19f530abf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 11:05:15.703068 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-60c6ccb4-0f64-4b22-9e88-7821814f13ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:05:15.718917 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 931/1851] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 11:05:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 248 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 26 11:05:15.855578 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:DELETE=7,keystone:SELECT=91,keystone:INSERT=2 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 11:05:15.904738 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 921/1852] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 11:05:15 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 246 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 11:05:15.909328 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5e2caa-c6ad-4e6c-8496-af2c8afe77d2 service neutron] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:05:15.920734 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5e2caa-c6ad-4e6c-8496-af2c8afe77d2 service neutron] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:05:15.996238 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab225350-9dfa-458a-8f4f-9e1eb6d860cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:05:15.996866 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab225350-9dfa-458a-8f4f-9e1eb6d860cd admin admin] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:05:15.997128 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab225350-9dfa-458a-8f4f-9e1eb6d860cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:05:15.997674 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab225350-9dfa-458a-8f4f-9e1eb6d860cd admin admin] PATH_INFO: `/v3/projects` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:05:15.998849 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab225350-9dfa-458a-8f4f-9e1eb6d860cd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:05:16.001064 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab225350-9dfa-458a-8f4f-9e1eb6d860cd admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:05:16.037297 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c6ccb4-0f64-4b22-9e88-7821814f13ce None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:05:16.040445 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-60c6ccb4-0f64-4b22-9e88-7821814f13ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:05:16.060791 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c6ccb4-0f64-4b22-9e88-7821814f13ce None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:05:16.066017 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 932/1853] 10.222.0.40 () {60 vars in 1153 bytes} [Fri Aug 26 11:05:15 2022] POST /identity/v3/projects => generated 387 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 26 11:05:16.071138 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab225350-9dfa-458a-8f4f-9e1eb6d860cd admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:05:16.073394 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4718d9-e957-4bd2-9b39-922f36426930 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:05:16.075812 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4718d9-e957-4bd2-9b39-922f36426930 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:05:16.076447 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4718d9-e957-4bd2-9b39-922f36426930 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:05:16.078079 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4718d9-e957-4bd2-9b39-922f36426930 admin admin] PATH_INFO: `/v3/users/84c54f3982934872b2cf688a5f9507ef` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:05:16.082705 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e4718d9-e957-4bd2-9b39-922f36426930 admin admin] RBAC: Authorizing `identity:delete_user(user_id=84c54f3982934872b2cf688a5f9507ef)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:05:16.086434 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 11:05:16.087788 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e4718d9-e957-4bd2-9b39-922f36426930 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:05:16.167476 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab225350-9dfa-458a-8f4f-9e1eb6d860cd admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:05:16.174398 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea466b45-82d8-43b1-9b3e-93f241f82419 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:05:16.175337 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea466b45-82d8-43b1-9b3e-93f241f82419 admin admin] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:05:16.175817 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea466b45-82d8-43b1-9b3e-93f241f82419 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:05:16.176346 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea466b45-82d8-43b1-9b3e-93f241f82419 admin admin] PATH_INFO: `/v3/users` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:05:16.177223 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea466b45-82d8-43b1-9b3e-93f241f82419 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:05:16.180806 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea466b45-82d8-43b1-9b3e-93f241f82419 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:05:16.215245 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-2e4718d9-e957-4bd2-9b39-922f36426930 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '84c54f3982934872b2cf688a5f9507ef'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:05:16.228924 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 933/1854] 10.222.0.40 () {60 vars in 1144 bytes} [Fri Aug 26 11:05:16 2022] POST /identity/v3/users => generated 300 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 11:05:16.228924 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea466b45-82d8-43b1-9b3e-93f241f82419 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:05:16.235209 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-2e4718d9-e957-4bd2-9b39-922f36426930 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '84c54f3982934872b2cf688a5f9507ef'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:05:16.237166 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-2e4718d9-e957-4bd2-9b39-922f36426930 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '84c54f3982934872b2cf688a5f9507ef'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:05:16.244433 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea466b45-82d8-43b1-9b3e-93f241f82419 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:05:16.255742 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-2e4718d9-e957-4bd2-9b39-922f36426930 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '84c54f3982934872b2cf688a5f9507ef'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:05:16.257470 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462f5fb8-ba90-4525-b8f3-078facb27918 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:05:16.258181 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462f5fb8-ba90-4525-b8f3-078facb27918 admin admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:05:16.258461 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462f5fb8-ba90-4525-b8f3-078facb27918 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:05:16.258734 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462f5fb8-ba90-4525-b8f3-078facb27918 admin admin] PATH_INFO: `/v3/roles` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:05:16.259517 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-462f5fb8-ba90-4525-b8f3-078facb27918 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:05:16.262810 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-462f5fb8-ba90-4525-b8f3-078facb27918 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:05:16.263406 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-2e4718d9-e957-4bd2-9b39-922f36426930 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '84c54f3982934872b2cf688a5f9507ef'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:05:16.278828 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 934/1855] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 26 11:05:16 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 11:05:16.280694 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462f5fb8-ba90-4525-b8f3-078facb27918 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:05:16.283091 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-462f5fb8-ba90-4525-b8f3-078facb27918 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:05:16.292003 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 922/1856] 10.222.0.40 () {58 vars in 1224 bytes} [Fri Aug 26 11:05:16 2022] DELETE /identity/v3/users/84c54f3982934872b2cf688a5f9507ef => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 11:05:16.299601 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba546b93-4865-4473-a3e8-980664d3e6c3 None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:05:16.301447 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba546b93-4865-4473-a3e8-980664d3e6c3 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:05:16.301699 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba546b93-4865-4473-a3e8-980664d3e6c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:05:16.315616 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462f5fb8-ba90-4525-b8f3-078facb27918 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:05:16.339814 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-ba546b93-4865-4473-a3e8-980664d3e6c3 None None] MFA Rules not processed for user `ca155a48e2024c32af3fd2e19e5fac11`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 11:05:16.341771 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-ba546b93-4865-4473-a3e8-980664d3e6c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:05:16.478347 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22950a0b-54d3-479b-9eb8-aca12351a11b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:05:16.481278 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22950a0b-54d3-479b-9eb8-aca12351a11b admin admin] REQUEST_METHOD: `PUT` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:05:16.481582 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22950a0b-54d3-479b-9eb8-aca12351a11b admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:05:16.481862 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22950a0b-54d3-479b-9eb8-aca12351a11b admin admin] PATH_INFO: `/v3/projects/9f49d0bd1a7c41d68b405b12310bd912/users/f4659194af494350958aa40d12889b5e/roles/1f97e4602e2e4d6498913a81f9ae8d6a` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:05:16.502266 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 923/1857] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 11:05:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 11:05:16.524921 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22950a0b-54d3-479b-9eb8-aca12351a11b admin admin] RBAC: Authorizing `identity:create_grant(project_id=9f49d0bd1a7c41d68b405b12310bd912, user_id=f4659194af494350958aa40d12889b5e, role_id=1f97e4602e2e4d6498913a81f9ae8d6a)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:05:16.527655 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22950a0b-54d3-479b-9eb8-aca12351a11b admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:05:16.546691 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba546b93-4865-4473-a3e8-980664d3e6c3 None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:05:16.550763 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-ba546b93-4865-4473-a3e8-980664d3e6c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:05:16.559914 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 935/1858] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 26 11:05:16 2022] PUT /identity/v3/projects/9f49d0bd1a7c41d68b405b12310bd912/users/f4659194af494350958aa40d12889b5e/roles/1f97e4602e2e4d6498913a81f9ae8d6a => generated 0 bytes in 280 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 11:05:16.564564 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22950a0b-54d3-479b-9eb8-aca12351a11b admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:05:16.684495 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba546b93-4865-4473-a3e8-980664d3e6c3 None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:05:16.718691 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22950a0b-54d3-479b-9eb8-aca12351a11b admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:05:16.725952 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b07a2a9-e40c-4691-b536-6f53ef9bb2f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:05:16.726793 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b07a2a9-e40c-4691-b536-6f53ef9bb2f3 admin admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:05:16.727170 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b07a2a9-e40c-4691-b536-6f53ef9bb2f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:05:16.727564 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b07a2a9-e40c-4691-b536-6f53ef9bb2f3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:05:16.728884 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b07a2a9-e40c-4691-b536-6f53ef9bb2f3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:05:16.729615 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 11:05:16.733470 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b07a2a9-e40c-4691-b536-6f53ef9bb2f3 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:05:16.757229 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 936/1859] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 26 11:05:16 2022] GET /identity/v3/roles => generated 1420 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 11:05:16.761369 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b07a2a9-e40c-4691-b536-6f53ef9bb2f3 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:05:16.769377 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 11:05:16.783250 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b07a2a9-e40c-4691-b536-6f53ef9bb2f3 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:05:16.796204 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3332ba-6ddd-40c8-85af-943220257de5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:05:16.796967 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3332ba-6ddd-40c8-85af-943220257de5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:05:16.797250 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3332ba-6ddd-40c8-85af-943220257de5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:05:16.797548 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3332ba-6ddd-40c8-85af-943220257de5 admin admin] PATH_INFO: `/v3/projects/9f49d0bd1a7c41d68b405b12310bd912/users/f4659194af494350958aa40d12889b5e/roles/1f97e4602e2e4d6498913a81f9ae8d6a` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:05:16.813511 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d3332ba-6ddd-40c8-85af-943220257de5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9f49d0bd1a7c41d68b405b12310bd912, user_id=f4659194af494350958aa40d12889b5e, role_id=1f97e4602e2e4d6498913a81f9ae8d6a)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:05:16.818023 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d3332ba-6ddd-40c8-85af-943220257de5 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:05:16.833683 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92285525-7657-415d-a55a-c6f9e3bf85b8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91da2acd2534423091537e1ba068d2b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e30ccf0c1204727b786876c141dbcd2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:05:16.835529 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92285525-7657-415d-a55a-c6f9e3bf85b8 service neutron] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:05:16.836046 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92285525-7657-415d-a55a-c6f9e3bf85b8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:05:16.837158 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92285525-7657-415d-a55a-c6f9e3bf85b8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:05:16.862252 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 937/1860] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 26 11:05:16 2022] PUT /identity/v3/projects/9f49d0bd1a7c41d68b405b12310bd912/users/f4659194af494350958aa40d12889b5e/roles/1f97e4602e2e4d6498913a81f9ae8d6a => generated 0 bytes in 91 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 26 11:05:16.863038 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92285525-7657-415d-a55a-c6f9e3bf85b8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:05:16.865123 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92285525-7657-415d-a55a-c6f9e3bf85b8 service neutron] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:05:16.866116 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e01fbc6-7bc5-4cbb-a632-f03f06a89a94 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:05:16.868508 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e01fbc6-7bc5-4cbb-a632-f03f06a89a94 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:05:16.868904 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e01fbc6-7bc5-4cbb-a632-f03f06a89a94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:05:16.899358 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 924/1861] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 11:05:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 11:05:16.934296 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-7e01fbc6-7bc5-4cbb-a632-f03f06a89a94 None None] MFA Rules not processed for user `f4659194af494350958aa40d12889b5e`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 11:05:16.935660 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-7e01fbc6-7bc5-4cbb-a632-f03f06a89a94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:05:17.044769 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 938/1862] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 11:05:16 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 11:05:17.058325 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92285525-7657-415d-a55a-c6f9e3bf85b8 service neutron] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:05:17.059936 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-92285525-7657-415d-a55a-c6f9e3bf85b8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:05:17.162151 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92285525-7657-415d-a55a-c6f9e3bf85b8 service neutron] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:05:17.183308 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e01fbc6-7bc5-4cbb-a632-f03f06a89a94 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:05:17.184903 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-7e01fbc6-7bc5-4cbb-a632-f03f06a89a94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:05:17.345071 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caebf7be-819d-4593-8d5f-29ac0146c01e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c277338a7f814cc6b01c5419775031bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e30ccf0c1204727b786876c141dbcd2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:05:17.345910 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caebf7be-819d-4593-8d5f-29ac0146c01e service cinder] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:05:17.346808 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caebf7be-819d-4593-8d5f-29ac0146c01e service cinder] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:05:17.347247 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caebf7be-819d-4593-8d5f-29ac0146c01e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:05:17.373475 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caebf7be-819d-4593-8d5f-29ac0146c01e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:05:17.383852 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caebf7be-819d-4593-8d5f-29ac0146c01e service cinder] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:05:17.404069 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e01fbc6-7bc5-4cbb-a632-f03f06a89a94 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:05:17.410963 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684fe182-ad3e-43a3-a2c2-a05809476f2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:05:17.411730 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684fe182-ad3e-43a3-a2c2-a05809476f2b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:05:17.412068 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684fe182-ad3e-43a3-a2c2-a05809476f2b admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:05:17.412349 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684fe182-ad3e-43a3-a2c2-a05809476f2b admin admin] PATH_INFO: `/v3/projects/d196d50f5be14178a60b08107ea0813d` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:05:17.414446 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-684fe182-ad3e-43a3-a2c2-a05809476f2b admin admin] RBAC: Authorizing `identity:delete_project(project_id=d196d50f5be14178a60b08107ea0813d)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:05:17.416887 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-684fe182-ad3e-43a3-a2c2-a05809476f2b admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:05:17.417117 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 925/1863] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 11:05:17 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 360 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 11:05:17.501827 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-684fe182-ad3e-43a3-a2c2-a05809476f2b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd196d50f5be14178a60b08107ea0813d', 'request_id': 'req-684fe182-ad3e-43a3-a2c2-a05809476f2b'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:05:17.523648 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-684fe182-ad3e-43a3-a2c2-a05809476f2b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd196d50f5be14178a60b08107ea0813d', 'request_id': 'req-684fe182-ad3e-43a3-a2c2-a05809476f2b'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:05:17.539808 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-684fe182-ad3e-43a3-a2c2-a05809476f2b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd196d50f5be14178a60b08107ea0813d', 'request_id': 'req-684fe182-ad3e-43a3-a2c2-a05809476f2b'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:05:17.563848 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-684fe182-ad3e-43a3-a2c2-a05809476f2b admin admin] The token cache is being invalidate because project d196d50f5be14178a60b08107ea0813d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81424) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 11:05:17.564080 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-684fe182-ad3e-43a3-a2c2-a05809476f2b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:05:17.564732 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-684fe182-ad3e-43a3-a2c2-a05809476f2b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:05:17.567330 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 939/1864] 10.222.0.40 () {58 vars in 1233 bytes} [Fri Aug 26 11:05:17 2022] DELETE /identity/v3/projects/d196d50f5be14178a60b08107ea0813d => generated 0 bytes in 384 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 11:05:17.571691 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caebf7be-819d-4593-8d5f-29ac0146c01e service cinder] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:05:17.588451 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caebf7be-819d-4593-8d5f-29ac0146c01e service cinder] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:05:17.626313 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684fe182-ad3e-43a3-a2c2-a05809476f2b admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:05:17.628638 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-684fe182-ad3e-43a3-a2c2-a05809476f2b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:05:17.636397 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 11:05:17.791781 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684fe182-ad3e-43a3-a2c2-a05809476f2b admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:05:17.806340 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e19bb5-7e2e-4ef8-a875-5ccd325066cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:05:17.808101 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e19bb5-7e2e-4ef8-a875-5ccd325066cc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:05:17.810089 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e19bb5-7e2e-4ef8-a875-5ccd325066cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:05:17.810562 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2111780f-5f48-43c6-a714-aea2e984141c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:05:17.810859 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e19bb5-7e2e-4ef8-a875-5ccd325066cc admin admin] PATH_INFO: `/v3/users/97e0a839a2d541efb7529d19f530abf9` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:05:17.811353 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2111780f-5f48-43c6-a714-aea2e984141c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:05:17.811755 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2111780f-5f48-43c6-a714-aea2e984141c admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:05:17.812106 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2111780f-5f48-43c6-a714-aea2e984141c admin admin] PATH_INFO: `/v3/users/c62dcaac2b714cbd9a8eb5ccfe93128e` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:05:17.813461 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2111780f-5f48-43c6-a714-aea2e984141c admin admin] RBAC: Authorizing `identity:delete_user(user_id=c62dcaac2b714cbd9a8eb5ccfe93128e)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:05:17.815544 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e19bb5-7e2e-4ef8-a875-5ccd325066cc admin admin] RBAC: Authorizing `identity:delete_user(user_id=97e0a839a2d541efb7529d19f530abf9)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:05:17.816005 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2111780f-5f48-43c6-a714-aea2e984141c admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:05:17.822497 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e19bb5-7e2e-4ef8-a875-5ccd325066cc admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:05:17.838281 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 11:05:17.921351 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-74e19bb5-7e2e-4ef8-a875-5ccd325066cc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '97e0a839a2d541efb7529d19f530abf9'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:05:17.924304 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-74e19bb5-7e2e-4ef8-a875-5ccd325066cc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '97e0a839a2d541efb7529d19f530abf9'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:05:17.924742 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-2111780f-5f48-43c6-a714-aea2e984141c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c62dcaac2b714cbd9a8eb5ccfe93128e'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:05:17.937812 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-2111780f-5f48-43c6-a714-aea2e984141c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c62dcaac2b714cbd9a8eb5ccfe93128e'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:05:17.940102 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-2111780f-5f48-43c6-a714-aea2e984141c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c62dcaac2b714cbd9a8eb5ccfe93128e'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:05:17.941504 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-74e19bb5-7e2e-4ef8-a875-5ccd325066cc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '97e0a839a2d541efb7529d19f530abf9'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:05:17.950323 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-74e19bb5-7e2e-4ef8-a875-5ccd325066cc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '97e0a839a2d541efb7529d19f530abf9'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:05:17.978114 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-2111780f-5f48-43c6-a714-aea2e984141c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c62dcaac2b714cbd9a8eb5ccfe93128e'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:05:17.979192 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-2111780f-5f48-43c6-a714-aea2e984141c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c62dcaac2b714cbd9a8eb5ccfe93128e'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:05:17.983930 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-74e19bb5-7e2e-4ef8-a875-5ccd325066cc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '97e0a839a2d541efb7529d19f530abf9'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:05:18.005183 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 926/1865] 10.222.0.40 () {58 vars in 1224 bytes} [Fri Aug 26 11:05:17 2022] DELETE /identity/v3/users/c62dcaac2b714cbd9a8eb5ccfe93128e => generated 0 bytes in 431 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 26 11:05:18.030715 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 940/1866] 10.222.0.40 () {58 vars in 1224 bytes} [Fri Aug 26 11:05:17 2022] DELETE /identity/v3/users/97e0a839a2d541efb7529d19f530abf9 => generated 0 bytes in 403 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 11:05:18.032806 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac32cd1-85e0-4c5a-9797-6cb90c7a1d10 None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:05:18.033507 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac32cd1-85e0-4c5a-9797-6cb90c7a1d10 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:05:18.034122 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac32cd1-85e0-4c5a-9797-6cb90c7a1d10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:05:18.065485 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-9ac32cd1-85e0-4c5a-9797-6cb90c7a1d10 None None] MFA Rules not processed for user `ca155a48e2024c32af3fd2e19e5fac11`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 11:05:18.067113 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-9ac32cd1-85e0-4c5a-9797-6cb90c7a1d10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:05:18.198465 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 927/1867] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 11:05:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 11:05:18.278862 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e19bb5-7e2e-4ef8-a875-5ccd325066cc admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:05:18.280537 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-74e19bb5-7e2e-4ef8-a875-5ccd325066cc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:05:18.384778 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e19bb5-7e2e-4ef8-a875-5ccd325066cc admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:05:18.390289 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5d840c-dc22-492f-ba86-0786a2368f6e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91da2acd2534423091537e1ba068d2b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e30ccf0c1204727b786876c141dbcd2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:05:18.391334 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5d840c-dc22-492f-ba86-0786a2368f6e service neutron] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:05:18.391981 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5d840c-dc22-492f-ba86-0786a2368f6e service neutron] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:05:18.392411 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5d840c-dc22-492f-ba86-0786a2368f6e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:05:18.422018 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d5d840c-dc22-492f-ba86-0786a2368f6e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:05:18.424215 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d5d840c-dc22-492f-ba86-0786a2368f6e service neutron] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:05:18.438286 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 941/1868] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 11:05:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 26 11:05:18.498636 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac32cd1-85e0-4c5a-9797-6cb90c7a1d10 None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:05:18.500506 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-9ac32cd1-85e0-4c5a-9797-6cb90c7a1d10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:05:18.517150 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac32cd1-85e0-4c5a-9797-6cb90c7a1d10 None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:05:18.525157 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c346542d-78c5-480d-aede-12098e2b4aed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:05:18.526079 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c346542d-78c5-480d-aede-12098e2b4aed admin admin] REQUEST_METHOD: `DELETE` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:05:18.526515 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c346542d-78c5-480d-aede-12098e2b4aed admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:05:18.526933 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c346542d-78c5-480d-aede-12098e2b4aed admin admin] PATH_INFO: `/v3/projects/a0cd6e51377248f58dcde39f5874200a` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:05:18.529557 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c346542d-78c5-480d-aede-12098e2b4aed admin admin] RBAC: Authorizing `identity:delete_project(project_id=a0cd6e51377248f58dcde39f5874200a)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:05:18.532349 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c346542d-78c5-480d-aede-12098e2b4aed admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:05:18.607408 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-c346542d-78c5-480d-aede-12098e2b4aed admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a0cd6e51377248f58dcde39f5874200a', 'request_id': 'req-c346542d-78c5-480d-aede-12098e2b4aed'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:05:18.615492 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-c346542d-78c5-480d-aede-12098e2b4aed admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a0cd6e51377248f58dcde39f5874200a', 'request_id': 'req-c346542d-78c5-480d-aede-12098e2b4aed'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:05:18.624114 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-c346542d-78c5-480d-aede-12098e2b4aed admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a0cd6e51377248f58dcde39f5874200a', 'request_id': 'req-c346542d-78c5-480d-aede-12098e2b4aed'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:05:18.641300 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-c346542d-78c5-480d-aede-12098e2b4aed admin admin] The token cache is being invalidate because project a0cd6e51377248f58dcde39f5874200a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81423) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 11:05:18.641751 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-c346542d-78c5-480d-aede-12098e2b4aed admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:05:18.642560 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-c346542d-78c5-480d-aede-12098e2b4aed admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:05:18.644571 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 928/1869] 10.222.0.40 () {58 vars in 1233 bytes} [Fri Aug 26 11:05:18 2022] DELETE /identity/v3/projects/a0cd6e51377248f58dcde39f5874200a => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 11:05:18.648943 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5d840c-dc22-492f-ba86-0786a2368f6e service neutron] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:05:18.660110 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5d840c-dc22-492f-ba86-0786a2368f6e service neutron] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:05:18.694681 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 11:05:18.749673 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7eddd98-3e1e-4fae-885a-125ed747224a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:05:18.751084 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7eddd98-3e1e-4fae-885a-125ed747224a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:05:18.751084 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7eddd98-3e1e-4fae-885a-125ed747224a admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:05:18.753318 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7eddd98-3e1e-4fae-885a-125ed747224a admin admin] PATH_INFO: `/v3/users/426c0f39ef3b473d8c1c881fd1babe2c` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:05:18.754678 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7eddd98-3e1e-4fae-885a-125ed747224a admin admin] RBAC: Authorizing `identity:delete_user(user_id=426c0f39ef3b473d8c1c881fd1babe2c)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:05:18.757715 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7eddd98-3e1e-4fae-885a-125ed747224a admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:05:18.822768 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-d7eddd98-3e1e-4fae-885a-125ed747224a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '426c0f39ef3b473d8c1c881fd1babe2c'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:05:18.823449 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-d7eddd98-3e1e-4fae-885a-125ed747224a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '426c0f39ef3b473d8c1c881fd1babe2c'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:05:18.833386 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-d7eddd98-3e1e-4fae-885a-125ed747224a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '426c0f39ef3b473d8c1c881fd1babe2c'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:05:18.834288 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-d7eddd98-3e1e-4fae-885a-125ed747224a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '426c0f39ef3b473d8c1c881fd1babe2c'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:05:18.852047 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-d7eddd98-3e1e-4fae-885a-125ed747224a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '426c0f39ef3b473d8c1c881fd1babe2c'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:05:18.895670 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 942/1870] 10.222.0.40 () {58 vars in 1224 bytes} [Fri Aug 26 11:05:18 2022] DELETE /identity/v3/users/426c0f39ef3b473d8c1c881fd1babe2c => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 26 11:05:19.467173 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c346542d-78c5-480d-aede-12098e2b4aed admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:05:19.468768 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-c346542d-78c5-480d-aede-12098e2b4aed admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:05:19.489088 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c346542d-78c5-480d-aede-12098e2b4aed admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:05:19.493041 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da14474a-fe33-47e4-b624-34e5a8549ed9 None None] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:05:19.493413 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da14474a-fe33-47e4-b624-34e5a8549ed9 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:05:19.493759 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da14474a-fe33-47e4-b624-34e5a8549ed9 None None] PATH_INFO: `/` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:05:19.495608 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 943/1871] 10.222.0.40 () {54 vars in 921 bytes} [Fri Aug 26 11:05:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 11:05:19.511870 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 11:05:19.513266 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb8e949-8e22-42f9-be58-e26dde462a24 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:05:19.513764 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb8e949-8e22-42f9-be58-e26dde462a24 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:05:19.513764 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb8e949-8e22-42f9-be58-e26dde462a24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:05:19.555054 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-afb8e949-8e22-42f9-be58-e26dde462a24 None None] MFA Rules not processed for user `1f4b4a662db049ecacc865ecff10bdb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 11:05:19.556640 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-afb8e949-8e22-42f9-be58-e26dde462a24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:05:19.596948 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba11d82e-9ccd-4870-8ce8-2ace6834c644 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:05:19.597735 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba11d82e-9ccd-4870-8ce8-2ace6834c644 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:05:19.599729 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba11d82e-9ccd-4870-8ce8-2ace6834c644 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:05:19.600542 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba11d82e-9ccd-4870-8ce8-2ace6834c644 admin admin] PATH_INFO: `/v3/projects/375a02448ae84e8f858713e6ac2c5dd6` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:05:19.601553 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba11d82e-9ccd-4870-8ce8-2ace6834c644 admin admin] RBAC: Authorizing `identity:delete_project(project_id=375a02448ae84e8f858713e6ac2c5dd6)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:05:19.614255 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba11d82e-9ccd-4870-8ce8-2ace6834c644 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:05:19.697774 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-ba11d82e-9ccd-4870-8ce8-2ace6834c644 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '375a02448ae84e8f858713e6ac2c5dd6', 'request_id': 'req-ba11d82e-9ccd-4870-8ce8-2ace6834c644'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:05:19.699441 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 944/1872] 10.222.0.40 () {58 vars in 1018 bytes} [Fri Aug 26 11:05:19 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 11:05:19.705599 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-ba11d82e-9ccd-4870-8ce8-2ace6834c644 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '375a02448ae84e8f858713e6ac2c5dd6', 'request_id': 'req-ba11d82e-9ccd-4870-8ce8-2ace6834c644'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:05:19.715248 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-ba11d82e-9ccd-4870-8ce8-2ace6834c644 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '375a02448ae84e8f858713e6ac2c5dd6', 'request_id': 'req-ba11d82e-9ccd-4870-8ce8-2ace6834c644'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:05:19.730870 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-ba11d82e-9ccd-4870-8ce8-2ace6834c644 admin admin] The token cache is being invalidate because project 375a02448ae84e8f858713e6ac2c5dd6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81423) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 11:05:19.731566 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-ba11d82e-9ccd-4870-8ce8-2ace6834c644 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:05:19.732977 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-ba11d82e-9ccd-4870-8ce8-2ace6834c644 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:05:19.734043 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 929/1873] 10.222.0.40 () {58 vars in 1233 bytes} [Fri Aug 26 11:05:19 2022] DELETE /identity/v3/projects/375a02448ae84e8f858713e6ac2c5dd6 => generated 0 bytes in 267 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 11:05:19.739479 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb8e949-8e22-42f9-be58-e26dde462a24 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:05:19.742468 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-afb8e949-8e22-42f9-be58-e26dde462a24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:05:19.759003 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb8e949-8e22-42f9-be58-e26dde462a24 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:05:19.808927 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba11d82e-9ccd-4870-8ce8-2ace6834c644 admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:05:19.809772 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 11:05:19.810645 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-ba11d82e-9ccd-4870-8ce8-2ace6834c644 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:05:19.854992 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996affbc-b6e5-4252-b69b-a61b340e55b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:05:19.855337 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996affbc-b6e5-4252-b69b-a61b340e55b5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:05:19.855854 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996affbc-b6e5-4252-b69b-a61b340e55b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:05:19.856297 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996affbc-b6e5-4252-b69b-a61b340e55b5 admin admin] PATH_INFO: `/v3/users/67ba6b1e818e49e0b0e2aaf18ba0ebf6` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:05:19.857746 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-996affbc-b6e5-4252-b69b-a61b340e55b5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=67ba6b1e818e49e0b0e2aaf18ba0ebf6)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:05:19.860000 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-996affbc-b6e5-4252-b69b-a61b340e55b5 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:05:19.945909 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-996affbc-b6e5-4252-b69b-a61b340e55b5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '67ba6b1e818e49e0b0e2aaf18ba0ebf6'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:05:19.947063 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-996affbc-b6e5-4252-b69b-a61b340e55b5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '67ba6b1e818e49e0b0e2aaf18ba0ebf6'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:05:19.962497 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba11d82e-9ccd-4870-8ce8-2ace6834c644 admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:05:19.970267 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-996affbc-b6e5-4252-b69b-a61b340e55b5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '67ba6b1e818e49e0b0e2aaf18ba0ebf6'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:05:19.972919 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-996affbc-b6e5-4252-b69b-a61b340e55b5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '67ba6b1e818e49e0b0e2aaf18ba0ebf6'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:05:19.973599 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578a6bf6-4782-4fbf-b9ec-8d08c2e986e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:05:19.974556 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578a6bf6-4782-4fbf-b9ec-8d08c2e986e9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:05:19.975981 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578a6bf6-4782-4fbf-b9ec-8d08c2e986e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:05:19.976486 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578a6bf6-4782-4fbf-b9ec-8d08c2e986e9 admin admin] PATH_INFO: `/v3/projects/c9cd769db7204342904cf450ba649e27` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:05:19.978627 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-578a6bf6-4782-4fbf-b9ec-8d08c2e986e9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c9cd769db7204342904cf450ba649e27)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:05:19.982853 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-578a6bf6-4782-4fbf-b9ec-8d08c2e986e9 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:05:19.994290 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-996affbc-b6e5-4252-b69b-a61b340e55b5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '67ba6b1e818e49e0b0e2aaf18ba0ebf6'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:05:20.022279 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 945/1874] 10.222.0.40 () {58 vars in 1224 bytes} [Fri Aug 26 11:05:19 2022] DELETE /identity/v3/users/67ba6b1e818e49e0b0e2aaf18ba0ebf6 => generated 0 bytes in 285 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 11:05:20.023391 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996affbc-b6e5-4252-b69b-a61b340e55b5 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:05:20.025062 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-996affbc-b6e5-4252-b69b-a61b340e55b5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:05:20.108306 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996affbc-b6e5-4252-b69b-a61b340e55b5 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:05:20.122978 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 11:05:20.126193 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-578a6bf6-4782-4fbf-b9ec-8d08c2e986e9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c9cd769db7204342904cf450ba649e27', 'request_id': 'req-578a6bf6-4782-4fbf-b9ec-8d08c2e986e9'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:05:20.135558 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-578a6bf6-4782-4fbf-b9ec-8d08c2e986e9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c9cd769db7204342904cf450ba649e27', 'request_id': 'req-578a6bf6-4782-4fbf-b9ec-8d08c2e986e9'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:05:20.145192 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-578a6bf6-4782-4fbf-b9ec-8d08c2e986e9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c9cd769db7204342904cf450ba649e27', 'request_id': 'req-578a6bf6-4782-4fbf-b9ec-8d08c2e986e9'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:05:20.168622 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-578a6bf6-4782-4fbf-b9ec-8d08c2e986e9 admin admin] The token cache is being invalidate because project c9cd769db7204342904cf450ba649e27 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81423) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 11:05:20.169381 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-578a6bf6-4782-4fbf-b9ec-8d08c2e986e9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:05:20.169932 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-578a6bf6-4782-4fbf-b9ec-8d08c2e986e9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:05:20.171850 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 930/1875] 10.222.0.40 () {58 vars in 1233 bytes} [Fri Aug 26 11:05:19 2022] DELETE /identity/v3/projects/c9cd769db7204342904cf450ba649e27 => generated 0 bytes in 363 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 26 11:05:20.174847 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578a6bf6-4782-4fbf-b9ec-8d08c2e986e9 admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:05:20.175137 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-578a6bf6-4782-4fbf-b9ec-8d08c2e986e9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:05:20.204887 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78cb5e1f-d0c0-4a85-a8bc-c19b2e3762d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:05:20.205439 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78cb5e1f-d0c0-4a85-a8bc-c19b2e3762d3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:05:20.205746 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78cb5e1f-d0c0-4a85-a8bc-c19b2e3762d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:05:20.206011 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78cb5e1f-d0c0-4a85-a8bc-c19b2e3762d3 admin admin] PATH_INFO: `/v3/projects/b9e12223e2714aa6b4bad666b07904af` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:05:20.208929 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78cb5e1f-d0c0-4a85-a8bc-c19b2e3762d3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b9e12223e2714aa6b4bad666b07904af)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:05:20.211984 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78cb5e1f-d0c0-4a85-a8bc-c19b2e3762d3 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:05:20.231657 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 11:05:20.306205 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578a6bf6-4782-4fbf-b9ec-8d08c2e986e9 admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:05:20.307910 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-78cb5e1f-d0c0-4a85-a8bc-c19b2e3762d3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b9e12223e2714aa6b4bad666b07904af', 'request_id': 'req-78cb5e1f-d0c0-4a85-a8bc-c19b2e3762d3'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:05:20.317424 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-78cb5e1f-d0c0-4a85-a8bc-c19b2e3762d3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b9e12223e2714aa6b4bad666b07904af', 'request_id': 'req-78cb5e1f-d0c0-4a85-a8bc-c19b2e3762d3'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:05:20.327053 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-78cb5e1f-d0c0-4a85-a8bc-c19b2e3762d3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b9e12223e2714aa6b4bad666b07904af', 'request_id': 'req-78cb5e1f-d0c0-4a85-a8bc-c19b2e3762d3'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:05:20.344657 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-78cb5e1f-d0c0-4a85-a8bc-c19b2e3762d3 admin admin] The token cache is being invalidate because project b9e12223e2714aa6b4bad666b07904af was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81424) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 11:05:20.345118 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-78cb5e1f-d0c0-4a85-a8bc-c19b2e3762d3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:05:20.345879 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-78cb5e1f-d0c0-4a85-a8bc-c19b2e3762d3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:05:20.347905 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 946/1876] 10.222.0.40 () {58 vars in 1233 bytes} [Fri Aug 26 11:05:20 2022] DELETE /identity/v3/projects/b9e12223e2714aa6b4bad666b07904af => generated 0 bytes in 325 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 11:05:20.348784 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78cb5e1f-d0c0-4a85-a8bc-c19b2e3762d3 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:05:20.350338 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-78cb5e1f-d0c0-4a85-a8bc-c19b2e3762d3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:05:20.398256 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c39742-3d5c-4969-a9fc-dbeef9cdbf89 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f4b4a662db049ecacc865ecff10bdb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e30ccf0c1204727b786876c141dbcd2', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:05:20.398999 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c39742-3d5c-4969-a9fc-dbeef9cdbf89 service nova] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:05:20.399373 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c39742-3d5c-4969-a9fc-dbeef9cdbf89 service nova] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:05:20.399648 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c39742-3d5c-4969-a9fc-dbeef9cdbf89 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:05:20.401933 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-97c39742-3d5c-4969-a9fc-dbeef9cdbf89 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:05:20.415727 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c39742-3d5c-4969-a9fc-dbeef9cdbf89 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:05:20.417911 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c39742-3d5c-4969-a9fc-dbeef9cdbf89 service nova] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:05:20.433779 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 931/1877] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 11:05:20 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 261 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 11:05:20.435285 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15873678-baf0-46f1-88ef-1670a0b54cf5 None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:05:20.435524 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78cb5e1f-d0c0-4a85-a8bc-c19b2e3762d3 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:05:20.435644 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15873678-baf0-46f1-88ef-1670a0b54cf5 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:05:20.436004 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15873678-baf0-46f1-88ef-1670a0b54cf5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:05:20.441235 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa83790-e35b-4a5e-b73d-4c34cfd35ce7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:05:20.443353 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa83790-e35b-4a5e-b73d-4c34cfd35ce7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:05:20.444584 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa83790-e35b-4a5e-b73d-4c34cfd35ce7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:05:20.445241 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa83790-e35b-4a5e-b73d-4c34cfd35ce7 admin admin] PATH_INFO: `/v3/users/f4659194af494350958aa40d12889b5e` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:05:20.446911 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa83790-e35b-4a5e-b73d-4c34cfd35ce7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f4659194af494350958aa40d12889b5e)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:05:20.448937 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa83790-e35b-4a5e-b73d-4c34cfd35ce7 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:05:20.462377 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-15873678-baf0-46f1-88ef-1670a0b54cf5 None None] MFA Rules not processed for user `ca155a48e2024c32af3fd2e19e5fac11`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 11:05:20.464183 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-15873678-baf0-46f1-88ef-1670a0b54cf5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:05:20.493827 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 932/1878] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 11:05:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 59 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 11:05:20.499330 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96784d15-1a0b-49ed-9fd2-f01214ec8f10 None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:05:20.500117 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96784d15-1a0b-49ed-9fd2-f01214ec8f10 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:05:20.500567 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96784d15-1a0b-49ed-9fd2-f01214ec8f10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:05:20.547248 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-efa83790-e35b-4a5e-b73d-4c34cfd35ce7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f4659194af494350958aa40d12889b5e'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:05:20.550168 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-efa83790-e35b-4a5e-b73d-4c34cfd35ce7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f4659194af494350958aa40d12889b5e'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:05:20.557339 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-96784d15-1a0b-49ed-9fd2-f01214ec8f10 None None] MFA Rules not processed for user `ca155a48e2024c32af3fd2e19e5fac11`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 11:05:20.559483 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-96784d15-1a0b-49ed-9fd2-f01214ec8f10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:05:20.582324 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-efa83790-e35b-4a5e-b73d-4c34cfd35ce7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f4659194af494350958aa40d12889b5e'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:05:20.585814 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-efa83790-e35b-4a5e-b73d-4c34cfd35ce7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f4659194af494350958aa40d12889b5e'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:05:20.605035 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 933/1879] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 11:05:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 11:05:20.612003 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96784d15-1a0b-49ed-9fd2-f01214ec8f10 None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:05:20.634733 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-efa83790-e35b-4a5e-b73d-4c34cfd35ce7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f4659194af494350958aa40d12889b5e'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:05:20.642363 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96784d15-1a0b-49ed-9fd2-f01214ec8f10 None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:05:20.656204 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796de75d-8730-4095-b25c-1e944fd1ea0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:05:20.657025 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796de75d-8730-4095-b25c-1e944fd1ea0a admin admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:05:20.657377 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796de75d-8730-4095-b25c-1e944fd1ea0a admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:05:20.657722 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796de75d-8730-4095-b25c-1e944fd1ea0a admin admin] PATH_INFO: `/v3/domains` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:05:20.659980 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-796de75d-8730-4095-b25c-1e944fd1ea0a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 11:05:20.659980 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-796de75d-8730-4095-b25c-1e944fd1ea0a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:05:20.664817 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-796de75d-8730-4095-b25c-1e944fd1ea0a admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:05:20.665945 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 947/1880] 10.222.0.40 () {58 vars in 1224 bytes} [Fri Aug 26 11:05:20 2022] DELETE /identity/v3/users/f4659194af494350958aa40d12889b5e => generated 0 bytes in 317 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 11:05:20.731151 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 934/1881] 10.222.0.40 () {58 vars in 1153 bytes} [Fri Aug 26 11:05:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 121 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 11:05:20.731925 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa83790-e35b-4a5e-b73d-4c34cfd35ce7 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:05:20.750518 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796de75d-8730-4095-b25c-1e944fd1ea0a admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:05:20.754078 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-796de75d-8730-4095-b25c-1e944fd1ea0a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:05:20.758272 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 11:05:20.783250 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796de75d-8730-4095-b25c-1e944fd1ea0a admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:05:20.833450 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa83790-e35b-4a5e-b73d-4c34cfd35ce7 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:05:20.841225 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5efec40-ab4f-4839-89f6-4ca783d45e45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:05:20.842641 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5efec40-ab4f-4839-89f6-4ca783d45e45 admin admin] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:05:20.843168 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=96,keystone:DELETE=3,keystone:INSERT=1 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 11:05:20.843603 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5efec40-ab4f-4839-89f6-4ca783d45e45 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:05:20.844024 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5efec40-ab4f-4839-89f6-4ca783d45e45 admin admin] PATH_INFO: `/v3/projects` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:05:20.844807 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5efec40-ab4f-4839-89f6-4ca783d45e45 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:05:20.848038 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5efec40-ab4f-4839-89f6-4ca783d45e45 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:05:20.898724 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 948/1882] 10.222.0.40 () {60 vars in 1153 bytes} [Fri Aug 26 11:05:20 2022] POST /identity/v3/projects => generated 385 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 11:05:20.903287 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5efec40-ab4f-4839-89f6-4ca783d45e45 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:05:20.910594 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e936b9e6-204b-45a9-95bb-048308e61280 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:05:20.914435 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e936b9e6-204b-45a9-95bb-048308e61280 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:05:20.915035 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e936b9e6-204b-45a9-95bb-048308e61280 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:05:20.915599 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e936b9e6-204b-45a9-95bb-048308e61280 admin admin] PATH_INFO: `/v3/projects/82f2d67519ee491386963c3b3969391e` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:05:20.923392 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e936b9e6-204b-45a9-95bb-048308e61280 admin admin] RBAC: Authorizing `identity:delete_project(project_id=82f2d67519ee491386963c3b3969391e)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:05:20.928365 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e936b9e6-204b-45a9-95bb-048308e61280 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:05:21.039838 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-e936b9e6-204b-45a9-95bb-048308e61280 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '82f2d67519ee491386963c3b3969391e', 'request_id': 'req-e936b9e6-204b-45a9-95bb-048308e61280'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:05:21.051591 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-e936b9e6-204b-45a9-95bb-048308e61280 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '82f2d67519ee491386963c3b3969391e', 'request_id': 'req-e936b9e6-204b-45a9-95bb-048308e61280'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:05:21.071051 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-e936b9e6-204b-45a9-95bb-048308e61280 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '82f2d67519ee491386963c3b3969391e', 'request_id': 'req-e936b9e6-204b-45a9-95bb-048308e61280'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:05:21.074950 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5efec40-ab4f-4839-89f6-4ca783d45e45 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:05:21.090974 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-e936b9e6-204b-45a9-95bb-048308e61280 admin admin] The token cache is being invalidate because project 82f2d67519ee491386963c3b3969391e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81423) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 11:05:21.091408 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-e936b9e6-204b-45a9-95bb-048308e61280 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:05:21.092583 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-e936b9e6-204b-45a9-95bb-048308e61280 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:05:21.094884 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 935/1883] 10.222.0.40 () {58 vars in 1233 bytes} [Fri Aug 26 11:05:20 2022] DELETE /identity/v3/projects/82f2d67519ee491386963c3b3969391e => generated 0 bytes in 345 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 26 11:05:21.106089 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be390e5-d863-4c7c-be09-d6a288cfde2c None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:05:21.106680 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be390e5-d863-4c7c-be09-d6a288cfde2c None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:05:21.107095 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be390e5-d863-4c7c-be09-d6a288cfde2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:05:21.135534 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-7be390e5-d863-4c7c-be09-d6a288cfde2c None None] MFA Rules not processed for user `ca155a48e2024c32af3fd2e19e5fac11`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 11:05:21.138401 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-7be390e5-d863-4c7c-be09-d6a288cfde2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:05:21.232655 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 936/1884] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 11:05:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 11:05:21.234649 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49525f6-19b4-4e53-b86b-6a7a1fe7b39c None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:05:21.234751 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49525f6-19b4-4e53-b86b-6a7a1fe7b39c None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:05:21.234981 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49525f6-19b4-4e53-b86b-6a7a1fe7b39c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:05:21.256719 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45338dbb-c7ce-43ea-b87f-dd038d1f30d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:05:21.257663 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45338dbb-c7ce-43ea-b87f-dd038d1f30d7 admin admin] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:05:21.258005 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45338dbb-c7ce-43ea-b87f-dd038d1f30d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:05:21.258288 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45338dbb-c7ce-43ea-b87f-dd038d1f30d7 admin admin] PATH_INFO: `/v3/users` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:05:21.258921 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45338dbb-c7ce-43ea-b87f-dd038d1f30d7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:05:21.260980 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45338dbb-c7ce-43ea-b87f-dd038d1f30d7 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:05:21.267610 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-a49525f6-19b4-4e53-b86b-6a7a1fe7b39c None None] MFA Rules not processed for user `ca155a48e2024c32af3fd2e19e5fac11`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 11:05:21.270174 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-a49525f6-19b4-4e53-b86b-6a7a1fe7b39c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:05:21.295910 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 937/1885] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 11:05:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 59 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 11:05:21.299881 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49525f6-19b4-4e53-b86b-6a7a1fe7b39c None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:05:21.326410 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 949/1886] 10.222.0.40 () {60 vars in 1144 bytes} [Fri Aug 26 11:05:20 2022] POST /identity/v3/users => generated 293 bytes in 424 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 11:05:21.330279 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45338dbb-c7ce-43ea-b87f-dd038d1f30d7 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:05:21.332313 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49525f6-19b4-4e53-b86b-6a7a1fe7b39c None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:05:21.333483 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-45338dbb-c7ce-43ea-b87f-dd038d1f30d7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:05:21.338650 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4544119d-d30e-4ec6-86e8-8c4882fd7eeb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:05:21.339588 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4544119d-d30e-4ec6-86e8-8c4882fd7eeb admin admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:05:21.340077 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4544119d-d30e-4ec6-86e8-8c4882fd7eeb admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:05:21.340669 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4544119d-d30e-4ec6-86e8-8c4882fd7eeb admin admin] PATH_INFO: `/v3/domains` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:05:21.341380 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4544119d-d30e-4ec6-86e8-8c4882fd7eeb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 11:05:21.341629 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4544119d-d30e-4ec6-86e8-8c4882fd7eeb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:05:21.343310 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4544119d-d30e-4ec6-86e8-8c4882fd7eeb admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:05:21.354227 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45338dbb-c7ce-43ea-b87f-dd038d1f30d7 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:05:21.360487 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 938/1887] 10.222.0.40 () {58 vars in 1153 bytes} [Fri Aug 26 11:05:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 11:05:21.361497 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4544119d-d30e-4ec6-86e8-8c4882fd7eeb admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:05:21.363777 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f541f93b-6cb6-4275-9a38-2ef02b16e555 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:05:21.364653 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f541f93b-6cb6-4275-9a38-2ef02b16e555 admin admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:05:21.365132 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f541f93b-6cb6-4275-9a38-2ef02b16e555 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:05:21.366324 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f541f93b-6cb6-4275-9a38-2ef02b16e555 admin admin] PATH_INFO: `/v3/roles` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:05:21.367489 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f541f93b-6cb6-4275-9a38-2ef02b16e555 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:05:21.372842 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4544119d-d30e-4ec6-86e8-8c4882fd7eeb admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:05:21.377877 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f541f93b-6cb6-4275-9a38-2ef02b16e555 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:05:21.378560 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0d412c-a596-41d7-aa8e-03a4f0153678 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:05:21.379449 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0d412c-a596-41d7-aa8e-03a4f0153678 admin admin] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:05:21.379735 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0d412c-a596-41d7-aa8e-03a4f0153678 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:05:21.380006 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0d412c-a596-41d7-aa8e-03a4f0153678 admin admin] PATH_INFO: `/v3/projects` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:05:21.380774 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea0d412c-a596-41d7-aa8e-03a4f0153678 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:05:21.383276 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea0d412c-a596-41d7-aa8e-03a4f0153678 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:05:21.397415 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 950/1888] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 26 11:05:21 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 26 11:05:21.399163 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f541f93b-6cb6-4275-9a38-2ef02b16e555 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:05:21.414972 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f541f93b-6cb6-4275-9a38-2ef02b16e555 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:05:21.421714 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a6c3ce-47b3-4b99-8b0e-414a913cf5c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:05:21.422742 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a6c3ce-47b3-4b99-8b0e-414a913cf5c3 admin admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:05:21.423286 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a6c3ce-47b3-4b99-8b0e-414a913cf5c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:05:21.423800 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a6c3ce-47b3-4b99-8b0e-414a913cf5c3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:05:21.424651 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a6c3ce-47b3-4b99-8b0e-414a913cf5c3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:05:21.426829 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a6c3ce-47b3-4b99-8b0e-414a913cf5c3 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:05:21.445739 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 939/1889] 10.222.0.40 () {60 vars in 1153 bytes} [Fri Aug 26 11:05:21 2022] POST /identity/v3/projects => generated 365 bytes in 81 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 26 11:05:21.449295 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0d412c-a596-41d7-aa8e-03a4f0153678 admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:05:21.449935 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 951/1890] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 26 11:05:21 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 11:05:21.451437 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a6c3ce-47b3-4b99-8b0e-414a913cf5c3 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:05:21.465268 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a6c3ce-47b3-4b99-8b0e-414a913cf5c3 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:05:21.478251 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 11:05:21.508836 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 11:05:21.594571 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0d412c-a596-41d7-aa8e-03a4f0153678 admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:05:21.599889 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc98122-5c6a-4291-9755-627f53ac4bde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:05:21.600609 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84391f94-6df1-4583-bf44-26cfb9e530e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:05:21.601353 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc98122-5c6a-4291-9755-627f53ac4bde admin admin] REQUEST_METHOD: `PUT` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:05:21.601483 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84391f94-6df1-4583-bf44-26cfb9e530e9 admin admin] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:05:21.601646 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84391f94-6df1-4583-bf44-26cfb9e530e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:05:21.601921 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc98122-5c6a-4291-9755-627f53ac4bde admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:05:21.602065 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84391f94-6df1-4583-bf44-26cfb9e530e9 admin admin] PATH_INFO: `/v3/users` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:05:21.602406 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc98122-5c6a-4291-9755-627f53ac4bde admin admin] PATH_INFO: `/v3/projects/4491a5e616ff4d1bb9d46a372efa4997/users/fa828a2dedf4475db9e229f0d1851439/roles/23e174ce743b46eabe618c212691da55` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:05:21.602768 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84391f94-6df1-4583-bf44-26cfb9e530e9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:05:21.605110 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84391f94-6df1-4583-bf44-26cfb9e530e9 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:05:21.624415 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc98122-5c6a-4291-9755-627f53ac4bde admin admin] RBAC: Authorizing `identity:create_grant(project_id=4491a5e616ff4d1bb9d46a372efa4997, user_id=fa828a2dedf4475db9e229f0d1851439, role_id=23e174ce743b46eabe618c212691da55)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:05:21.628457 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc98122-5c6a-4291-9755-627f53ac4bde admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:05:21.649808 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 940/1891] 10.222.0.40 () {60 vars in 1144 bytes} [Fri Aug 26 11:05:21 2022] POST /identity/v3/users => generated 283 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 11:05:21.653818 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84391f94-6df1-4583-bf44-26cfb9e530e9 admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:05:21.659859 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 952/1892] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 26 11:05:21 2022] PUT /identity/v3/projects/4491a5e616ff4d1bb9d46a372efa4997/users/fa828a2dedf4475db9e229f0d1851439/roles/23e174ce743b46eabe618c212691da55 => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 11:05:21.672291 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a748f7-20d2-4f51-ae58-a3b61098e133 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:05:21.672578 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a748f7-20d2-4f51-ae58-a3b61098e133 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:05:21.672853 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a748f7-20d2-4f51-ae58-a3b61098e133 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:05:21.679854 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84391f94-6df1-4583-bf44-26cfb9e530e9 admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:05:21.749601 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-c8a748f7-20d2-4f51-ae58-a3b61098e133 None None] MFA Rules not processed for user `fa828a2dedf4475db9e229f0d1851439`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 11:05:21.751896 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-c8a748f7-20d2-4f51-ae58-a3b61098e133 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:05:21.811100 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254355af-d038-423a-bc13-afb0425bfc37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:05:21.811928 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254355af-d038-423a-bc13-afb0425bfc37 admin admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:05:21.812461 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254355af-d038-423a-bc13-afb0425bfc37 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:05:21.812889 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254355af-d038-423a-bc13-afb0425bfc37 admin admin] PATH_INFO: `/v3/roles` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:05:21.813822 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-254355af-d038-423a-bc13-afb0425bfc37 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:05:21.815933 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-254355af-d038-423a-bc13-afb0425bfc37 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:05:21.828749 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 941/1893] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 26 11:05:21 2022] GET /identity/v3/roles => generated 1420 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 11:05:21.841844 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254355af-d038-423a-bc13-afb0425bfc37 admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:05:21.853575 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 953/1894] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 11:05:21 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 11:05:21.883578 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a748f7-20d2-4f51-ae58-a3b61098e133 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:05:21.884291 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-c8a748f7-20d2-4f51-ae58-a3b61098e133 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:05:21.884810 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254355af-d038-423a-bc13-afb0425bfc37 admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:05:21.894889 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d5b935-ffe2-4887-ab81-970ef69b2683 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:05:21.895620 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d5b935-ffe2-4887-ab81-970ef69b2683 admin admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:05:21.895960 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d5b935-ffe2-4887-ab81-970ef69b2683 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:05:21.896302 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d5b935-ffe2-4887-ab81-970ef69b2683 admin admin] PATH_INFO: `/v3/roles` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:05:21.897203 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d5b935-ffe2-4887-ab81-970ef69b2683 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:05:21.899164 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d5b935-ffe2-4887-ab81-970ef69b2683 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:05:21.911018 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 942/1895] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 26 11:05:21 2022] GET /identity/v3/roles => generated 1420 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 11:05:21.917768 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d5b935-ffe2-4887-ab81-970ef69b2683 admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:05:21.930029 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d5b935-ffe2-4887-ab81-970ef69b2683 admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:05:21.935487 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f454f32-b289-4b8c-8a26-719949c7ecf6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:05:21.936142 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f454f32-b289-4b8c-8a26-719949c7ecf6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:05:21.936416 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f454f32-b289-4b8c-8a26-719949c7ecf6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:05:21.936817 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f454f32-b289-4b8c-8a26-719949c7ecf6 admin admin] PATH_INFO: `/v3/projects/08b7d78012a947cb9f1b78933fa94b83/users/8d1b7fe6be55401690e910148258a90a/roles/23e174ce743b46eabe618c212691da55` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:05:21.949373 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a748f7-20d2-4f51-ae58-a3b61098e133 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:05:21.953424 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3371cc29-752f-42d6-82fc-c0ca33b35fc6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c277338a7f814cc6b01c5419775031bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e30ccf0c1204727b786876c141dbcd2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:05:21.953894 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f454f32-b289-4b8c-8a26-719949c7ecf6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=08b7d78012a947cb9f1b78933fa94b83, user_id=8d1b7fe6be55401690e910148258a90a, role_id=23e174ce743b46eabe618c212691da55)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:05:21.954096 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3371cc29-752f-42d6-82fc-c0ca33b35fc6 service cinder] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:05:21.954341 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3371cc29-752f-42d6-82fc-c0ca33b35fc6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:05:21.954615 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3371cc29-752f-42d6-82fc-c0ca33b35fc6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:05:21.956841 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f454f32-b289-4b8c-8a26-719949c7ecf6 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:05:21.965707 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3371cc29-752f-42d6-82fc-c0ca33b35fc6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:05:21.967373 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3371cc29-752f-42d6-82fc-c0ca33b35fc6 service cinder] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:05:21.978799 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 954/1896] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 11:05:21 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 26 11:05:21.985276 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 943/1897] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 26 11:05:21 2022] PUT /identity/v3/projects/08b7d78012a947cb9f1b78933fa94b83/users/8d1b7fe6be55401690e910148258a90a/roles/23e174ce743b46eabe618c212691da55 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 11:05:22.001568 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834013fd-05e8-448f-a241-d0ab39557221 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:05:22.001863 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834013fd-05e8-448f-a241-d0ab39557221 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:05:22.002143 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834013fd-05e8-448f-a241-d0ab39557221 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:05:22.013745 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f454f32-b289-4b8c-8a26-719949c7ecf6 admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:05:22.047322 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-834013fd-05e8-448f-a241-d0ab39557221 None None] MFA Rules not processed for user `8d1b7fe6be55401690e910148258a90a`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 11:05:22.048569 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-834013fd-05e8-448f-a241-d0ab39557221 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:05:22.074387 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 11:05:22.128624 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f454f32-b289-4b8c-8a26-719949c7ecf6 admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:05:22.134706 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c5cb2d-1dcf-44c6-ab9a-59fede5c7d6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:05:22.135540 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c5cb2d-1dcf-44c6-ab9a-59fede5c7d6f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:05:22.135974 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c5cb2d-1dcf-44c6-ab9a-59fede5c7d6f admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:05:22.136409 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c5cb2d-1dcf-44c6-ab9a-59fede5c7d6f admin admin] PATH_INFO: `/v3/users/fa828a2dedf4475db9e229f0d1851439` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:05:22.137588 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c5cb2d-1dcf-44c6-ab9a-59fede5c7d6f admin admin] RBAC: Authorizing `identity:delete_user(user_id=fa828a2dedf4475db9e229f0d1851439)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:05:22.139628 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c5cb2d-1dcf-44c6-ab9a-59fede5c7d6f admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:05:22.144076 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 955/1898] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 11:05:21 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 11:05:22.146239 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834013fd-05e8-448f-a241-d0ab39557221 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:05:22.149292 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-834013fd-05e8-448f-a241-d0ab39557221 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:05:22.160340 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=95,keystone:INSERT=4,keystone:DELETE=1 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 11:05:22.168522 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834013fd-05e8-448f-a241-d0ab39557221 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:05:22.174303 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b87d410-fc35-4826-9959-8a89526b54ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:05:22.175594 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b87d410-fc35-4826-9959-8a89526b54ce admin admin] REQUEST_METHOD: `DELETE` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:05:22.176518 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b87d410-fc35-4826-9959-8a89526b54ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:05:22.177484 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b87d410-fc35-4826-9959-8a89526b54ce admin admin] PATH_INFO: `/v3/projects/9f49d0bd1a7c41d68b405b12310bd912` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:05:22.179922 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b87d410-fc35-4826-9959-8a89526b54ce admin admin] RBAC: Authorizing `identity:delete_project(project_id=9f49d0bd1a7c41d68b405b12310bd912)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:05:22.182134 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b87d410-fc35-4826-9959-8a89526b54ce admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:05:22.224309 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-38c5cb2d-1dcf-44c6-ab9a-59fede5c7d6f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fa828a2dedf4475db9e229f0d1851439'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:05:22.244588 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-38c5cb2d-1dcf-44c6-ab9a-59fede5c7d6f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fa828a2dedf4475db9e229f0d1851439'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:05:22.246587 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-38c5cb2d-1dcf-44c6-ab9a-59fede5c7d6f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fa828a2dedf4475db9e229f0d1851439'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:05:22.265905 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-2b87d410-fc35-4826-9959-8a89526b54ce admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9f49d0bd1a7c41d68b405b12310bd912', 'request_id': 'req-2b87d410-fc35-4826-9959-8a89526b54ce'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:05:22.272672 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-2b87d410-fc35-4826-9959-8a89526b54ce admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9f49d0bd1a7c41d68b405b12310bd912', 'request_id': 'req-2b87d410-fc35-4826-9959-8a89526b54ce'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:05:22.274355 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-38c5cb2d-1dcf-44c6-ab9a-59fede5c7d6f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fa828a2dedf4475db9e229f0d1851439'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:05:22.275114 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-38c5cb2d-1dcf-44c6-ab9a-59fede5c7d6f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fa828a2dedf4475db9e229f0d1851439'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:05:22.279096 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-2b87d410-fc35-4826-9959-8a89526b54ce admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9f49d0bd1a7c41d68b405b12310bd912', 'request_id': 'req-2b87d410-fc35-4826-9959-8a89526b54ce'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:05:22.297059 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 944/1899] 10.222.0.40 () {58 vars in 1224 bytes} [Fri Aug 26 11:05:22 2022] DELETE /identity/v3/users/fa828a2dedf4475db9e229f0d1851439 => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 11:05:22.297832 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-2b87d410-fc35-4826-9959-8a89526b54ce admin admin] The token cache is being invalidate because project 9f49d0bd1a7c41d68b405b12310bd912 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81424) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 11:05:22.298121 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c5cb2d-1dcf-44c6-ab9a-59fede5c7d6f admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:05:22.298236 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-2b87d410-fc35-4826-9959-8a89526b54ce admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:05:22.299156 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-2b87d410-fc35-4826-9959-8a89526b54ce admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:05:22.299675 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-38c5cb2d-1dcf-44c6-ab9a-59fede5c7d6f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:05:22.300888 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 956/1900] 10.222.0.40 () {58 vars in 1233 bytes} [Fri Aug 26 11:05:22 2022] DELETE /identity/v3/projects/9f49d0bd1a7c41d68b405b12310bd912 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 11:05:22.302140 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc95b25-8dc1-4bde-81a4-fdf2242a99cd None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:05:22.302426 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc95b25-8dc1-4bde-81a4-fdf2242a99cd None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:05:22.302699 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc95b25-8dc1-4bde-81a4-fdf2242a99cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:05:22.324268 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-bcc95b25-8dc1-4bde-81a4-fdf2242a99cd None None] MFA Rules not processed for user `ca155a48e2024c32af3fd2e19e5fac11`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 11:05:22.325532 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-bcc95b25-8dc1-4bde-81a4-fdf2242a99cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:05:22.374097 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c5cb2d-1dcf-44c6-ab9a-59fede5c7d6f admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:05:22.390784 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a5e32c-5c7d-4aed-a3fa-92d7106582d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:05:22.391324 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a5e32c-5c7d-4aed-a3fa-92d7106582d3 admin admin] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:05:22.391605 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a5e32c-5c7d-4aed-a3fa-92d7106582d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:05:22.391880 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a5e32c-5c7d-4aed-a3fa-92d7106582d3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:05:22.392812 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a5e32c-5c7d-4aed-a3fa-92d7106582d3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:05:22.397353 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a5e32c-5c7d-4aed-a3fa-92d7106582d3 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:05:22.421388 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 957/1901] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 11:05:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 11:05:22.423629 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09fc513-c366-4f10-bf5a-36dc31d014a5 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:05:22.424215 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09fc513-c366-4f10-bf5a-36dc31d014a5 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:05:22.425529 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09fc513-c366-4f10-bf5a-36dc31d014a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:05:22.450015 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-e09fc513-c366-4f10-bf5a-36dc31d014a5 None None] MFA Rules not processed for user `ca155a48e2024c32af3fd2e19e5fac11`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 11:05:22.451480 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-e09fc513-c366-4f10-bf5a-36dc31d014a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:05:22.466019 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 958/1902] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 11:05:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 11:05:22.466019 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09fc513-c366-4f10-bf5a-36dc31d014a5 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:05:22.471145 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-e09fc513-c366-4f10-bf5a-36dc31d014a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:05:22.474791 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 945/1903] 10.222.0.40 () {60 vars in 1153 bytes} [Fri Aug 26 11:05:22 2022] POST /identity/v3/projects => generated 365 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 11:05:22.479468 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ca98a1-42f3-4769-8d2d-8c9d5c7768af None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:05:22.479861 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ca98a1-42f3-4769-8d2d-8c9d5c7768af None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:05:22.480245 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ca98a1-42f3-4769-8d2d-8c9d5c7768af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:05:22.503296 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-48ca98a1-42f3-4769-8d2d-8c9d5c7768af None None] MFA Rules not processed for user `ca155a48e2024c32af3fd2e19e5fac11`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 11:05:22.504706 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-48ca98a1-42f3-4769-8d2d-8c9d5c7768af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:05:22.536512 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 11:05:22.583486 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09fc513-c366-4f10-bf5a-36dc31d014a5 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:05:22.655536 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 946/1904] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 11:05:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 11:05:22.660534 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ca98a1-42f3-4769-8d2d-8c9d5c7768af None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:05:22.664679 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-48ca98a1-42f3-4769-8d2d-8c9d5c7768af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:05:22.678840 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46f92d3-22f9-4fc3-942f-c97b45ebc57d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91da2acd2534423091537e1ba068d2b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e30ccf0c1204727b786876c141dbcd2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:05:22.679617 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46f92d3-22f9-4fc3-942f-c97b45ebc57d service neutron] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:05:22.679999 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46f92d3-22f9-4fc3-942f-c97b45ebc57d service neutron] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:05:22.680371 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46f92d3-22f9-4fc3-942f-c97b45ebc57d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:05:22.687159 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ca98a1-42f3-4769-8d2d-8c9d5c7768af None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:05:22.694741 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596a2f0a-1763-4734-bc8e-d5468131c39e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:05:22.695701 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b46f92d3-22f9-4fc3-942f-c97b45ebc57d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:05:22.696416 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596a2f0a-1763-4734-bc8e-d5468131c39e admin admin] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:05:22.698009 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b46f92d3-22f9-4fc3-942f-c97b45ebc57d service neutron] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:05:22.698377 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596a2f0a-1763-4734-bc8e-d5468131c39e admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:05:22.698751 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596a2f0a-1763-4734-bc8e-d5468131c39e admin admin] PATH_INFO: `/v3/users` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:05:22.699698 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-596a2f0a-1763-4734-bc8e-d5468131c39e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:05:22.706264 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-596a2f0a-1763-4734-bc8e-d5468131c39e admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:05:22.724531 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 959/1905] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 11:05:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 262 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 11:05:22.725813 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46f92d3-22f9-4fc3-942f-c97b45ebc57d service neutron] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:05:22.738659 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46f92d3-22f9-4fc3-942f-c97b45ebc57d service neutron] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:05:22.745437 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=86,keystone:DELETE=9,keystone:INSERT=5 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 11:05:22.757334 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 947/1906] 10.222.0.40 () {60 vars in 1144 bytes} [Fri Aug 26 11:05:22 2022] POST /identity/v3/users => generated 289 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 11:05:22.758249 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596a2f0a-1763-4734-bc8e-d5468131c39e admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:05:22.766294 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d63bd1-0421-4ad4-b96f-ba34a0e1b016 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:05:22.767119 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d63bd1-0421-4ad4-b96f-ba34a0e1b016 admin admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:05:22.768666 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d63bd1-0421-4ad4-b96f-ba34a0e1b016 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:05:22.771085 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d63bd1-0421-4ad4-b96f-ba34a0e1b016 admin admin] PATH_INFO: `/v3/domains` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:05:22.772161 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d63bd1-0421-4ad4-b96f-ba34a0e1b016 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 11:05:22.774465 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d63bd1-0421-4ad4-b96f-ba34a0e1b016 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:05:22.776059 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d63bd1-0421-4ad4-b96f-ba34a0e1b016 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:05:22.782396 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596a2f0a-1763-4734-bc8e-d5468131c39e admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:05:22.797566 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc0aa0e-c509-4343-8718-6c96b2618a91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:05:22.798983 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc0aa0e-c509-4343-8718-6c96b2618a91 admin admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:05:22.799723 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc0aa0e-c509-4343-8718-6c96b2618a91 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:05:22.800071 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc0aa0e-c509-4343-8718-6c96b2618a91 admin admin] PATH_INFO: `/v3/roles` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:05:22.801046 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc0aa0e-c509-4343-8718-6c96b2618a91 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:05:22.804861 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc0aa0e-c509-4343-8718-6c96b2618a91 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:05:22.813910 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 960/1907] 10.222.0.40 () {58 vars in 1153 bytes} [Fri Aug 26 11:05:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 11:05:22.815598 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d63bd1-0421-4ad4-b96f-ba34a0e1b016 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:05:22.835585 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 948/1908] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 26 11:05:22 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 11:05:22.838331 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d63bd1-0421-4ad4-b96f-ba34a0e1b016 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:05:22.843482 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc0aa0e-c509-4343-8718-6c96b2618a91 admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:05:22.844057 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa44eec-8d29-4b00-bf48-e1c9066c1c67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:05:22.845003 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa44eec-8d29-4b00-bf48-e1c9066c1c67 admin admin] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:05:22.845778 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa44eec-8d29-4b00-bf48-e1c9066c1c67 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:05:22.846281 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa44eec-8d29-4b00-bf48-e1c9066c1c67 admin admin] PATH_INFO: `/v3/projects` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:05:22.847255 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa44eec-8d29-4b00-bf48-e1c9066c1c67 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:05:22.852369 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa44eec-8d29-4b00-bf48-e1c9066c1c67 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:05:22.888848 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc0aa0e-c509-4343-8718-6c96b2618a91 admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:05:22.905850 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 961/1909] 10.222.0.40 () {60 vars in 1153 bytes} [Fri Aug 26 11:05:22 2022] POST /identity/v3/projects => generated 379 bytes in 87 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 11:05:22.907918 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beafc02d-20a6-4e54-86e4-228499e8d3fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:05:22.908610 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beafc02d-20a6-4e54-86e4-228499e8d3fe admin admin] REQUEST_METHOD: `PUT` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:05:22.908894 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beafc02d-20a6-4e54-86e4-228499e8d3fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:05:22.909189 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beafc02d-20a6-4e54-86e4-228499e8d3fe admin admin] PATH_INFO: `/v3/projects/1641c7f2ffe04631945584457d74c441/users/b99bd5e828a347749ecb96b9c17004ce/roles/1f97e4602e2e4d6498913a81f9ae8d6a` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:05:22.910107 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa44eec-8d29-4b00-bf48-e1c9066c1c67 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:05:22.960168 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beafc02d-20a6-4e54-86e4-228499e8d3fe admin admin] RBAC: Authorizing `identity:create_grant(project_id=1641c7f2ffe04631945584457d74c441, user_id=b99bd5e828a347749ecb96b9c17004ce, role_id=1f97e4602e2e4d6498913a81f9ae8d6a)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:05:22.963309 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beafc02d-20a6-4e54-86e4-228499e8d3fe admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:05:22.993577 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 949/1910] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 26 11:05:22 2022] PUT /identity/v3/projects/1641c7f2ffe04631945584457d74c441/users/b99bd5e828a347749ecb96b9c17004ce/roles/1f97e4602e2e4d6498913a81f9ae8d6a => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 11:05:22.998974 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beafc02d-20a6-4e54-86e4-228499e8d3fe admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:05:23.033407 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa44eec-8d29-4b00-bf48-e1c9066c1c67 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:05:23.130905 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beafc02d-20a6-4e54-86e4-228499e8d3fe admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:05:23.145604 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02daf430-1e95-4e66-bf24-30e39fa58018 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:05:23.146614 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02daf430-1e95-4e66-bf24-30e39fa58018 admin admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:05:23.147738 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02daf430-1e95-4e66-bf24-30e39fa58018 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:05:23.149177 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02daf430-1e95-4e66-bf24-30e39fa58018 admin admin] PATH_INFO: `/v3/roles` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:05:23.150763 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02daf430-1e95-4e66-bf24-30e39fa58018 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:05:23.155494 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02daf430-1e95-4e66-bf24-30e39fa58018 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:05:23.162775 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223e8c13-4069-42b6-a956-f315628fc8d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:05:23.163857 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223e8c13-4069-42b6-a956-f315628fc8d9 admin admin] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:05:23.164685 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223e8c13-4069-42b6-a956-f315628fc8d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:05:23.165308 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223e8c13-4069-42b6-a956-f315628fc8d9 admin admin] PATH_INFO: `/v3/users` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:05:23.166085 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-223e8c13-4069-42b6-a956-f315628fc8d9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:05:23.169276 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-223e8c13-4069-42b6-a956-f315628fc8d9 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:05:23.180477 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 950/1911] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 26 11:05:22 2022] GET /identity/v3/roles => generated 1420 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 11:05:23.185481 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02daf430-1e95-4e66-bf24-30e39fa58018 admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:05:23.186627 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 11:05:23.212829 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02daf430-1e95-4e66-bf24-30e39fa58018 admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:05:23.216848 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 962/1912] 10.222.0.40 () {60 vars in 1144 bytes} [Fri Aug 26 11:05:22 2022] POST /identity/v3/users => generated 290 bytes in 307 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 11:05:23.221305 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223e8c13-4069-42b6-a956-f315628fc8d9 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:05:23.228065 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d971efbe-60fd-4ba1-9cee-77016983bb45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:05:23.228764 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d971efbe-60fd-4ba1-9cee-77016983bb45 admin admin] REQUEST_METHOD: `PUT` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:05:23.229023 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d971efbe-60fd-4ba1-9cee-77016983bb45 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:05:23.229296 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d971efbe-60fd-4ba1-9cee-77016983bb45 admin admin] PATH_INFO: `/v3/projects/1641c7f2ffe04631945584457d74c441/users/b99bd5e828a347749ecb96b9c17004ce/roles/1f97e4602e2e4d6498913a81f9ae8d6a` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:05:23.234280 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d971efbe-60fd-4ba1-9cee-77016983bb45 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1641c7f2ffe04631945584457d74c441, user_id=b99bd5e828a347749ecb96b9c17004ce, role_id=1f97e4602e2e4d6498913a81f9ae8d6a)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:05:23.240196 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d971efbe-60fd-4ba1-9cee-77016983bb45 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:05:23.244878 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223e8c13-4069-42b6-a956-f315628fc8d9 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:05:23.252740 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f25f435-aa3e-4337-bf51-39ac0e92f703 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:05:23.253369 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f25f435-aa3e-4337-bf51-39ac0e92f703 admin admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:05:23.253648 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f25f435-aa3e-4337-bf51-39ac0e92f703 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:05:23.253922 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f25f435-aa3e-4337-bf51-39ac0e92f703 admin admin] PATH_INFO: `/v3/roles` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:05:23.254599 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f25f435-aa3e-4337-bf51-39ac0e92f703 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:05:23.256746 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f25f435-aa3e-4337-bf51-39ac0e92f703 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:05:23.279055 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 951/1913] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 26 11:05:23 2022] PUT /identity/v3/projects/1641c7f2ffe04631945584457d74c441/users/b99bd5e828a347749ecb96b9c17004ce/roles/1f97e4602e2e4d6498913a81f9ae8d6a => generated 0 bytes in 87 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 11:05:23.279055 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d971efbe-60fd-4ba1-9cee-77016983bb45 admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:05:23.279055 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-d971efbe-60fd-4ba1-9cee-77016983bb45 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:05:23.301238 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 963/1914] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 26 11:05:23 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 11:05:23.302850 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a12b3ab-ee71-4a6b-ab92-52b83f9a3ac9 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:05:23.303847 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a12b3ab-ee71-4a6b-ab92-52b83f9a3ac9 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:05:23.304126 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a12b3ab-ee71-4a6b-ab92-52b83f9a3ac9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:05:23.336035 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-4a12b3ab-ee71-4a6b-ab92-52b83f9a3ac9 None None] MFA Rules not processed for user `b99bd5e828a347749ecb96b9c17004ce`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 11:05:23.337441 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-4a12b3ab-ee71-4a6b-ab92-52b83f9a3ac9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:05:23.378565 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d971efbe-60fd-4ba1-9cee-77016983bb45 admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:05:23.388508 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59a68e2-154c-4ef1-823f-837f0879bfd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:05:23.389230 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59a68e2-154c-4ef1-823f-837f0879bfd7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:05:23.390005 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59a68e2-154c-4ef1-823f-837f0879bfd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:05:23.390368 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59a68e2-154c-4ef1-823f-837f0879bfd7 admin admin] PATH_INFO: `/v3/projects/4491a5e616ff4d1bb9d46a372efa4997` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:05:23.392018 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b59a68e2-154c-4ef1-823f-837f0879bfd7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4491a5e616ff4d1bb9d46a372efa4997)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:05:23.394445 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b59a68e2-154c-4ef1-823f-837f0879bfd7 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:05:23.441163 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 964/1915] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 11:05:23 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 11:05:23.442227 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a12b3ab-ee71-4a6b-ab92-52b83f9a3ac9 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:05:23.454730 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a12b3ab-ee71-4a6b-ab92-52b83f9a3ac9 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:05:23.467850 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-b59a68e2-154c-4ef1-823f-837f0879bfd7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4491a5e616ff4d1bb9d46a372efa4997', 'request_id': 'req-b59a68e2-154c-4ef1-823f-837f0879bfd7'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:05:23.469147 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=94,keystone:INSERT=2,keystone:DELETE=4 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 11:05:23.482498 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-b59a68e2-154c-4ef1-823f-837f0879bfd7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4491a5e616ff4d1bb9d46a372efa4997', 'request_id': 'req-b59a68e2-154c-4ef1-823f-837f0879bfd7'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:05:23.489681 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-b59a68e2-154c-4ef1-823f-837f0879bfd7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4491a5e616ff4d1bb9d46a372efa4997', 'request_id': 'req-b59a68e2-154c-4ef1-823f-837f0879bfd7'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:05:23.503599 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-b59a68e2-154c-4ef1-823f-837f0879bfd7 admin admin] The token cache is being invalidate because project 4491a5e616ff4d1bb9d46a372efa4997 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81423) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 11:05:23.503886 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-b59a68e2-154c-4ef1-823f-837f0879bfd7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:05:23.504986 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-b59a68e2-154c-4ef1-823f-837f0879bfd7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:05:23.506110 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 952/1916] 10.222.0.40 () {58 vars in 1233 bytes} [Fri Aug 26 11:05:23 2022] DELETE /identity/v3/projects/4491a5e616ff4d1bb9d46a372efa4997 => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 11:05:23.507613 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59a68e2-154c-4ef1-823f-837f0879bfd7 admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:05:23.508945 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-b59a68e2-154c-4ef1-823f-837f0879bfd7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:05:23.531100 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc00c72e-ef24-48d6-852d-5dcb645d3f7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:05:23.531721 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc00c72e-ef24-48d6-852d-5dcb645d3f7d admin admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:05:23.531997 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc00c72e-ef24-48d6-852d-5dcb645d3f7d admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:05:23.532265 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc00c72e-ef24-48d6-852d-5dcb645d3f7d admin admin] PATH_INFO: `/v3/roles` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:05:23.532909 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc00c72e-ef24-48d6-852d-5dcb645d3f7d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:05:23.534533 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc00c72e-ef24-48d6-852d-5dcb645d3f7d admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:05:23.546058 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 965/1917] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 26 11:05:23 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 11:05:23.547590 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73407a1-dd06-43aa-bf8a-43e1d208707f None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:05:23.547877 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73407a1-dd06-43aa-bf8a-43e1d208707f None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:05:23.548149 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73407a1-dd06-43aa-bf8a-43e1d208707f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:05:23.569187 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-e73407a1-dd06-43aa-bf8a-43e1d208707f None None] MFA Rules not processed for user `ca155a48e2024c32af3fd2e19e5fac11`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 11:05:23.570473 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-e73407a1-dd06-43aa-bf8a-43e1d208707f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:05:23.581197 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59a68e2-154c-4ef1-823f-837f0879bfd7 admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:05:23.583512 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 966/1918] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 11:05:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 11:05:23.584344 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73407a1-dd06-43aa-bf8a-43e1d208707f None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:05:23.585637 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ae0bda-f4c8-4681-b66d-cabb6805c638 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c277338a7f814cc6b01c5419775031bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e30ccf0c1204727b786876c141dbcd2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:05:23.585829 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-e73407a1-dd06-43aa-bf8a-43e1d208707f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:05:23.586401 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ae0bda-f4c8-4681-b66d-cabb6805c638 service cinder] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:05:23.586749 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ae0bda-f4c8-4681-b66d-cabb6805c638 service cinder] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:05:23.587182 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ae0bda-f4c8-4681-b66d-cabb6805c638 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:05:23.588979 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-38ae0bda-f4c8-4681-b66d-cabb6805c638 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:05:23.597843 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73407a1-dd06-43aa-bf8a-43e1d208707f None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:05:23.603489 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b71630-ccd5-47bc-baa3-a5f73bf686dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:05:23.604137 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b71630-ccd5-47bc-baa3-a5f73bf686dc admin admin] REQUEST_METHOD: `PUT` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:05:23.604411 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b71630-ccd5-47bc-baa3-a5f73bf686dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:05:23.604680 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b71630-ccd5-47bc-baa3-a5f73bf686dc admin admin] PATH_INFO: `/v3/projects/0dfb6a01448a4a128b1f6fe65532f9e8/users/7bfa3999c64e4e3b9e6fd420e7e6ed9a/roles/23e174ce743b46eabe618c212691da55` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:05:23.620829 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b71630-ccd5-47bc-baa3-a5f73bf686dc admin admin] RBAC: Authorizing `identity:create_grant(project_id=0dfb6a01448a4a128b1f6fe65532f9e8, user_id=7bfa3999c64e4e3b9e6fd420e7e6ed9a, role_id=23e174ce743b46eabe618c212691da55)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:05:23.624246 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b71630-ccd5-47bc-baa3-a5f73bf686dc admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:05:23.646037 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 967/1919] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 26 11:05:23 2022] PUT /identity/v3/projects/0dfb6a01448a4a128b1f6fe65532f9e8/users/7bfa3999c64e4e3b9e6fd420e7e6ed9a/roles/23e174ce743b46eabe618c212691da55 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 11:05:23.647896 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37462dc2-a45f-4363-a8c5-0820b6b16e35 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:05:23.648387 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37462dc2-a45f-4363-a8c5-0820b6b16e35 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:05:23.648819 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37462dc2-a45f-4363-a8c5-0820b6b16e35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:05:23.671403 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-37462dc2-a45f-4363-a8c5-0820b6b16e35 None None] MFA Rules not processed for user `ca155a48e2024c32af3fd2e19e5fac11`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 11:05:23.672625 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-37462dc2-a45f-4363-a8c5-0820b6b16e35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:05:23.684044 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ae0bda-f4c8-4681-b66d-cabb6805c638 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:05:23.685666 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ae0bda-f4c8-4681-b66d-cabb6805c638 service cinder] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:05:23.707368 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 953/1920] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 11:05:23 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 201 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 11:05:23.708902 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f71caea-ec49-4284-b0a1-b6b9b3f1f5ed None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:05:23.710082 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f71caea-ec49-4284-b0a1-b6b9b3f1f5ed None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:05:23.710414 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f71caea-ec49-4284-b0a1-b6b9b3f1f5ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:05:23.733048 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 11:05:23.776353 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-7f71caea-ec49-4284-b0a1-b6b9b3f1f5ed None None] MFA Rules not processed for user `7bfa3999c64e4e3b9e6fd420e7e6ed9a`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 11:05:23.777942 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-7f71caea-ec49-4284-b0a1-b6b9b3f1f5ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:05:23.789888 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 968/1921] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 11:05:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 11:05:23.791602 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37462dc2-a45f-4363-a8c5-0820b6b16e35 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:05:23.802917 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37462dc2-a45f-4363-a8c5-0820b6b16e35 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:05:23.812336 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27d85f4-15f7-4a64-9f7d-4dc8cfe38174 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:05:23.817097 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27d85f4-15f7-4a64-9f7d-4dc8cfe38174 admin admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:05:23.817425 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27d85f4-15f7-4a64-9f7d-4dc8cfe38174 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:05:23.817752 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27d85f4-15f7-4a64-9f7d-4dc8cfe38174 admin admin] PATH_INFO: `/v3/domains` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:05:23.818461 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f27d85f4-15f7-4a64-9f7d-4dc8cfe38174 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 11:05:23.819482 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f27d85f4-15f7-4a64-9f7d-4dc8cfe38174 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:05:23.824253 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f27d85f4-15f7-4a64-9f7d-4dc8cfe38174 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:05:23.859597 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 969/1922] 10.222.0.40 () {58 vars in 1153 bytes} [Fri Aug 26 11:05:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 11:05:23.859597 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27d85f4-15f7-4a64-9f7d-4dc8cfe38174 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:05:23.884933 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27d85f4-15f7-4a64-9f7d-4dc8cfe38174 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:05:23.891726 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 954/1923] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 11:05:23 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 11:05:23.891726 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f71caea-ec49-4284-b0a1-b6b9b3f1f5ed None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:05:23.897837 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bdc59d7-7015-4092-b6ba-2aed44ca6613 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:05:23.898585 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bdc59d7-7015-4092-b6ba-2aed44ca6613 admin admin] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:05:23.898941 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bdc59d7-7015-4092-b6ba-2aed44ca6613 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:05:23.899215 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bdc59d7-7015-4092-b6ba-2aed44ca6613 admin admin] PATH_INFO: `/v3/projects` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:05:23.899988 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bdc59d7-7015-4092-b6ba-2aed44ca6613 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:05:23.902105 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bdc59d7-7015-4092-b6ba-2aed44ca6613 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:05:23.912619 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=96,keystone:DELETE=3,keystone:INSERT=1 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 11:05:23.939566 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 970/1924] 10.222.0.40 () {60 vars in 1153 bytes} [Fri Aug 26 11:05:23 2022] POST /identity/v3/projects => generated 377 bytes in 83 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 11:05:23.941166 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bdc59d7-7015-4092-b6ba-2aed44ca6613 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:05:23.973263 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f71caea-ec49-4284-b0a1-b6b9b3f1f5ed None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:05:24.026941 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bdc59d7-7015-4092-b6ba-2aed44ca6613 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:05:24.035270 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497f3568-395f-41c2-947b-b3b04d04e1b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:05:24.036090 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f9ac63-140f-4fb0-9b67-85f88906a9f7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c277338a7f814cc6b01c5419775031bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e30ccf0c1204727b786876c141dbcd2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:05:24.036354 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497f3568-395f-41c2-947b-b3b04d04e1b4 admin admin] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:05:24.036726 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f9ac63-140f-4fb0-9b67-85f88906a9f7 service cinder] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:05:24.037016 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497f3568-395f-41c2-947b-b3b04d04e1b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:05:24.037138 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f9ac63-140f-4fb0-9b67-85f88906a9f7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:05:24.037292 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f9ac63-140f-4fb0-9b67-85f88906a9f7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:05:24.037949 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497f3568-395f-41c2-947b-b3b04d04e1b4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:05:24.039126 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-497f3568-395f-41c2-947b-b3b04d04e1b4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:05:24.039685 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-51f9ac63-140f-4fb0-9b67-85f88906a9f7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:05:24.041780 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-497f3568-395f-41c2-947b-b3b04d04e1b4 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:05:24.075918 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 971/1925] 10.222.0.40 () {60 vars in 1153 bytes} [Fri Aug 26 11:05:23 2022] POST /identity/v3/projects => generated 377 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 11:05:24.075918 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497f3568-395f-41c2-947b-b3b04d04e1b4 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:05:24.123198 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f9ac63-140f-4fb0-9b67-85f88906a9f7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:05:24.124647 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f9ac63-140f-4fb0-9b67-85f88906a9f7 service cinder] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:05:24.134608 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 955/1926] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 11:05:23 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 244 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 11:05:24.135483 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f9ac63-140f-4fb0-9b67-85f88906a9f7 service cinder] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:05:24.145858 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497f3568-395f-41c2-947b-b3b04d04e1b4 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:05:24.146414 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f9ac63-140f-4fb0-9b67-85f88906a9f7 service cinder] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:05:24.151348 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e835f77-9ff8-4419-8225-4040ca603cd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:05:24.152054 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e835f77-9ff8-4419-8225-4040ca603cd2 admin admin] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:05:24.152505 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e835f77-9ff8-4419-8225-4040ca603cd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:05:24.152611 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774a5f14-e181-4a1b-849c-162e4e56fca6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:05:24.152898 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e835f77-9ff8-4419-8225-4040ca603cd2 admin admin] PATH_INFO: `/v3/users` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:05:24.153374 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774a5f14-e181-4a1b-849c-162e4e56fca6 admin admin] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:05:24.153700 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e835f77-9ff8-4419-8225-4040ca603cd2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:05:24.153792 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774a5f14-e181-4a1b-849c-162e4e56fca6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:05:24.154139 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774a5f14-e181-4a1b-849c-162e4e56fca6 admin admin] PATH_INFO: `/v3/users` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:05:24.154832 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-774a5f14-e181-4a1b-849c-162e4e56fca6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:05:24.155896 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e835f77-9ff8-4419-8225-4040ca603cd2 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:05:24.156869 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-774a5f14-e181-4a1b-849c-162e4e56fca6 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:05:24.180974 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 972/1927] 10.222.0.40 () {60 vars in 1144 bytes} [Fri Aug 26 11:05:24 2022] POST /identity/v3/users => generated 289 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 11:05:24.185215 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e835f77-9ff8-4419-8225-4040ca603cd2 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:05:24.185832 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 956/1928] 10.222.0.40 () {60 vars in 1144 bytes} [Fri Aug 26 11:05:24 2022] POST /identity/v3/users => generated 289 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 11:05:24.189359 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774a5f14-e181-4a1b-849c-162e4e56fca6 admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:05:24.196130 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e835f77-9ff8-4419-8225-4040ca603cd2 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:05:24.199558 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774a5f14-e181-4a1b-849c-162e4e56fca6 admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:05:24.201053 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9eeadc-bd35-4658-a114-623400f220d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:05:24.201955 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9eeadc-bd35-4658-a114-623400f220d9 admin admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:05:24.201955 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9eeadc-bd35-4658-a114-623400f220d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:05:24.202255 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9eeadc-bd35-4658-a114-623400f220d9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:05:24.202916 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba9eeadc-bd35-4658-a114-623400f220d9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:05:24.204644 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba9eeadc-bd35-4658-a114-623400f220d9 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:05:24.204965 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f99031-32cc-438a-a73a-9dce2055bfdb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:05:24.205664 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f99031-32cc-438a-a73a-9dce2055bfdb admin admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:05:24.206006 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f99031-32cc-438a-a73a-9dce2055bfdb admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:05:24.206251 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f99031-32cc-438a-a73a-9dce2055bfdb admin admin] PATH_INFO: `/v3/roles` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:05:24.207014 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f99031-32cc-438a-a73a-9dce2055bfdb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:05:24.208452 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f99031-32cc-438a-a73a-9dce2055bfdb admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:05:24.215900 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 973/1929] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 26 11:05:24 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 26 11:05:24.219677 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9eeadc-bd35-4658-a114-623400f220d9 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:05:24.221784 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 957/1930] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 26 11:05:24 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 26 11:05:24.226516 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f99031-32cc-438a-a73a-9dce2055bfdb admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:05:24.226944 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 11:05:24.233725 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9eeadc-bd35-4658-a114-623400f220d9 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:05:24.237078 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f99031-32cc-438a-a73a-9dce2055bfdb admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:05:24.241401 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea61fea-87fb-457a-a702-5c4374022511 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:05:24.242326 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea61fea-87fb-457a-a702-5c4374022511 admin admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:05:24.242566 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7c3df3-ed26-4ddf-ba7a-ba2dac5fcd27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:05:24.242702 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea61fea-87fb-457a-a702-5c4374022511 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:05:24.242876 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea61fea-87fb-457a-a702-5c4374022511 admin admin] PATH_INFO: `/v3/roles` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:05:24.243606 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7c3df3-ed26-4ddf-ba7a-ba2dac5fcd27 admin admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:05:24.243715 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea61fea-87fb-457a-a702-5c4374022511 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:05:24.244057 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7c3df3-ed26-4ddf-ba7a-ba2dac5fcd27 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:05:24.244384 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7c3df3-ed26-4ddf-ba7a-ba2dac5fcd27 admin admin] PATH_INFO: `/v3/roles` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:05:24.245125 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa7c3df3-ed26-4ddf-ba7a-ba2dac5fcd27 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:05:24.245236 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea61fea-87fb-457a-a702-5c4374022511 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:05:24.246841 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa7c3df3-ed26-4ddf-ba7a-ba2dac5fcd27 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:05:24.257459 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 974/1931] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 26 11:05:24 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 26 11:05:24.258636 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 958/1932] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 26 11:05:24 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 26 11:05:24.261958 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea61fea-87fb-457a-a702-5c4374022511 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:05:24.262370 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7c3df3-ed26-4ddf-ba7a-ba2dac5fcd27 admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:05:24.273260 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea61fea-87fb-457a-a702-5c4374022511 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:05:24.274517 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7c3df3-ed26-4ddf-ba7a-ba2dac5fcd27 admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:05:24.279316 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6e6c49-6e9b-4eaf-9e6a-b05251f305be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:05:24.280066 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6e6c49-6e9b-4eaf-9e6a-b05251f305be admin admin] REQUEST_METHOD: `PUT` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:05:24.280353 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6e6c49-6e9b-4eaf-9e6a-b05251f305be admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:05:24.280683 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6e6c49-6e9b-4eaf-9e6a-b05251f305be admin admin] PATH_INFO: `/v3/projects/f37f73dcfccf49219b51fc4072114736/users/88ecd0cc5f2844ccadfe386e769f14f1/roles/23e174ce743b46eabe618c212691da55` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:05:24.287684 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d929da5d-bdbc-44d3-9beb-2bb1f0cbde21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:05:24.288544 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d929da5d-bdbc-44d3-9beb-2bb1f0cbde21 admin admin] REQUEST_METHOD: `PUT` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:05:24.288889 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d929da5d-bdbc-44d3-9beb-2bb1f0cbde21 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:05:24.289310 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d929da5d-bdbc-44d3-9beb-2bb1f0cbde21 admin admin] PATH_INFO: `/v3/projects/268541b52b674cb4a14e46236219cbdb/users/417c2e35727245fbbca6420395e50c44/roles/23e174ce743b46eabe618c212691da55` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:05:24.302809 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c6e6c49-6e9b-4eaf-9e6a-b05251f305be admin admin] RBAC: Authorizing `identity:create_grant(project_id=f37f73dcfccf49219b51fc4072114736, user_id=88ecd0cc5f2844ccadfe386e769f14f1, role_id=23e174ce743b46eabe618c212691da55)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:05:24.308495 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c6e6c49-6e9b-4eaf-9e6a-b05251f305be admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:05:24.311541 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d929da5d-bdbc-44d3-9beb-2bb1f0cbde21 admin admin] RBAC: Authorizing `identity:create_grant(project_id=268541b52b674cb4a14e46236219cbdb, user_id=417c2e35727245fbbca6420395e50c44, role_id=23e174ce743b46eabe618c212691da55)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:05:24.314479 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d929da5d-bdbc-44d3-9beb-2bb1f0cbde21 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:05:24.331977 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 975/1933] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 26 11:05:24 2022] PUT /identity/v3/projects/268541b52b674cb4a14e46236219cbdb/users/417c2e35727245fbbca6420395e50c44/roles/23e174ce743b46eabe618c212691da55 => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 11:05:24.343004 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b23c0c-b020-4759-be72-0fb8f004c3a9 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:05:24.343294 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b23c0c-b020-4759-be72-0fb8f004c3a9 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:05:24.343615 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b23c0c-b020-4759-be72-0fb8f004c3a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:05:24.346905 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 959/1934] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 26 11:05:24 2022] PUT /identity/v3/projects/f37f73dcfccf49219b51fc4072114736/users/88ecd0cc5f2844ccadfe386e769f14f1/roles/23e174ce743b46eabe618c212691da55 => generated 0 bytes in 85 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 26 11:05:24.358202 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6798d416-e63a-48a9-878c-2902e1594d23 None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:05:24.358490 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6798d416-e63a-48a9-878c-2902e1594d23 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:05:24.358768 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6798d416-e63a-48a9-878c-2902e1594d23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:05:24.394931 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-6798d416-e63a-48a9-878c-2902e1594d23 None None] MFA Rules not processed for user `88ecd0cc5f2844ccadfe386e769f14f1`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 11:05:24.396065 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-a8b23c0c-b020-4759-be72-0fb8f004c3a9 None None] MFA Rules not processed for user `417c2e35727245fbbca6420395e50c44`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 11:05:24.396198 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-6798d416-e63a-48a9-878c-2902e1594d23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:05:24.400087 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-a8b23c0c-b020-4759-be72-0fb8f004c3a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:05:24.416597 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 11:05:24.508825 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 960/1935] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 11:05:24 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 11:05:24.523163 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6798d416-e63a-48a9-878c-2902e1594d23 None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:05:24.538878 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 976/1936] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 11:05:24 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 11:05:24.554823 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b23c0c-b020-4759-be72-0fb8f004c3a9 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:05:24.671175 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b23c0c-b020-4759-be72-0fb8f004c3a9 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:05:24.676467 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3548444-f6fc-4fc9-bca6-17633220a8eb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c277338a7f814cc6b01c5419775031bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e30ccf0c1204727b786876c141dbcd2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:05:24.677094 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3548444-f6fc-4fc9-bca6-17633220a8eb service cinder] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:05:24.677373 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3548444-f6fc-4fc9-bca6-17633220a8eb service cinder] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:05:24.677644 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3548444-f6fc-4fc9-bca6-17633220a8eb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:05:24.683576 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6798d416-e63a-48a9-878c-2902e1594d23 None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:05:24.692572 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3548444-f6fc-4fc9-bca6-17633220a8eb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:05:24.694789 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3548444-f6fc-4fc9-bca6-17633220a8eb service cinder] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:05:24.705056 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 977/1937] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 11:05:24 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 11:05:24.708679 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d40a06-4469-4c5e-a030-cc5395246388 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:05:24.718894 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d40a06-4469-4c5e-a030-cc5395246388 admin admin] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:05:24.719466 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d40a06-4469-4c5e-a030-cc5395246388 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:05:24.719821 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d40a06-4469-4c5e-a030-cc5395246388 admin admin] PATH_INFO: `/v3/projects` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:05:24.720682 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d40a06-4469-4c5e-a030-cc5395246388 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:05:24.723245 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d40a06-4469-4c5e-a030-cc5395246388 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:05:24.785749 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 961/1938] 10.222.0.40 () {60 vars in 1153 bytes} [Fri Aug 26 11:05:24 2022] POST /identity/v3/projects => generated 379 bytes in 264 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 11:05:24.788126 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3548444-f6fc-4fc9-bca6-17633220a8eb service cinder] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:05:24.803607 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3548444-f6fc-4fc9-bca6-17633220a8eb service cinder] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:05:24.901105 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918ce737-49e8-4ed3-a7b0-7109cd0bb3fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:05:24.902220 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918ce737-49e8-4ed3-a7b0-7109cd0bb3fb admin admin] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:05:24.902689 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918ce737-49e8-4ed3-a7b0-7109cd0bb3fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:05:24.903060 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918ce737-49e8-4ed3-a7b0-7109cd0bb3fb admin admin] PATH_INFO: `/v3/users` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:05:24.904113 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-918ce737-49e8-4ed3-a7b0-7109cd0bb3fb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:05:24.906884 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-918ce737-49e8-4ed3-a7b0-7109cd0bb3fb admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:05:24.938034 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 11:05:24.966433 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 978/1939] 10.222.0.40 () {60 vars in 1144 bytes} [Fri Aug 26 11:05:24 2022] POST /identity/v3/users => generated 296 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 11:05:24.969256 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d40a06-4469-4c5e-a030-cc5395246388 admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:05:24.981337 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d40a06-4469-4c5e-a030-cc5395246388 admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:05:24.987592 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e23c55-463e-453e-a8c6-717df4680479 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:05:24.988216 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e23c55-463e-453e-a8c6-717df4680479 admin admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:05:24.988621 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e23c55-463e-453e-a8c6-717df4680479 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:05:24.988931 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e23c55-463e-453e-a8c6-717df4680479 admin admin] PATH_INFO: `/v3/roles` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:05:24.989661 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e23c55-463e-453e-a8c6-717df4680479 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:05:24.992585 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e23c55-463e-453e-a8c6-717df4680479 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:05:25.013955 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918ce737-49e8-4ed3-a7b0-7109cd0bb3fb admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:05:25.020703 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 962/1940] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 26 11:05:24 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 11:05:25.031698 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918ce737-49e8-4ed3-a7b0-7109cd0bb3fb admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:05:25.052022 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c497df5c-0370-498d-8d06-00a910b1c014 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:05:25.054434 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c497df5c-0370-498d-8d06-00a910b1c014 admin admin] REQUEST_METHOD: `PUT` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:05:25.055411 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c497df5c-0370-498d-8d06-00a910b1c014 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:05:25.055900 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c497df5c-0370-498d-8d06-00a910b1c014 admin admin] PATH_INFO: `/v3/projects/26fb5b128c6b48fd894424a4385f2e8f/users/d96ba4e7e8954058bdbaa6f64673ac4c/roles/1f97e4602e2e4d6498913a81f9ae8d6a` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:05:25.117429 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c497df5c-0370-498d-8d06-00a910b1c014 admin admin] RBAC: Authorizing `identity:create_grant(project_id=26fb5b128c6b48fd894424a4385f2e8f, user_id=d96ba4e7e8954058bdbaa6f64673ac4c, role_id=1f97e4602e2e4d6498913a81f9ae8d6a)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:05:25.120753 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c497df5c-0370-498d-8d06-00a910b1c014 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:05:25.145365 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 979/1941] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 26 11:05:25 2022] PUT /identity/v3/projects/26fb5b128c6b48fd894424a4385f2e8f/users/d96ba4e7e8954058bdbaa6f64673ac4c/roles/1f97e4602e2e4d6498913a81f9ae8d6a => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 26 11:05:25.149714 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e23c55-463e-453e-a8c6-717df4680479 admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:05:25.169790 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e23c55-463e-453e-a8c6-717df4680479 admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:05:25.273293 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5432218f-1fcf-497d-a774-dad1696fc222 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:05:25.274026 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5432218f-1fcf-497d-a774-dad1696fc222 admin admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:05:25.274383 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5432218f-1fcf-497d-a774-dad1696fc222 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:05:25.274724 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5432218f-1fcf-497d-a774-dad1696fc222 admin admin] PATH_INFO: `/v3/roles` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:05:25.275437 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5432218f-1fcf-497d-a774-dad1696fc222 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:05:25.277109 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5432218f-1fcf-497d-a774-dad1696fc222 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:05:25.288819 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 963/1942] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 26 11:05:25 2022] GET /identity/v3/roles => generated 1420 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 11:05:25.293098 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c497df5c-0370-498d-8d06-00a910b1c014 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:05:25.305335 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c497df5c-0370-498d-8d06-00a910b1c014 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:05:25.314907 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e98d2bd-e98b-4f73-8c3a-d3bd48bafaa1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:05:25.316187 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e98d2bd-e98b-4f73-8c3a-d3bd48bafaa1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:05:25.316687 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e98d2bd-e98b-4f73-8c3a-d3bd48bafaa1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:05:25.317297 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e98d2bd-e98b-4f73-8c3a-d3bd48bafaa1 admin admin] PATH_INFO: `/v3/projects/26fb5b128c6b48fd894424a4385f2e8f/users/d96ba4e7e8954058bdbaa6f64673ac4c/roles/1f97e4602e2e4d6498913a81f9ae8d6a` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:05:25.319982 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e98d2bd-e98b-4f73-8c3a-d3bd48bafaa1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=26fb5b128c6b48fd894424a4385f2e8f, user_id=d96ba4e7e8954058bdbaa6f64673ac4c, role_id=1f97e4602e2e4d6498913a81f9ae8d6a)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:05:25.323180 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e98d2bd-e98b-4f73-8c3a-d3bd48bafaa1 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:05:25.339815 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 980/1943] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 26 11:05:25 2022] PUT /identity/v3/projects/26fb5b128c6b48fd894424a4385f2e8f/users/d96ba4e7e8954058bdbaa6f64673ac4c/roles/1f97e4602e2e4d6498913a81f9ae8d6a => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 11:05:25.356100 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99de0a73-c431-434e-9f01-9c2520eb5990 None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:05:25.356396 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99de0a73-c431-434e-9f01-9c2520eb5990 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:05:25.357345 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99de0a73-c431-434e-9f01-9c2520eb5990 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:05:25.393014 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-99de0a73-c431-434e-9f01-9c2520eb5990 None None] MFA Rules not processed for user `d96ba4e7e8954058bdbaa6f64673ac4c`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 11:05:25.399304 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-99de0a73-c431-434e-9f01-9c2520eb5990 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:05:25.421334 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 11:05:25.481556 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 964/1944] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 11:05:25 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 11:05:25.494279 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e98d2bd-e98b-4f73-8c3a-d3bd48bafaa1 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:05:25.496027 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-3e98d2bd-e98b-4f73-8c3a-d3bd48bafaa1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:05:25.554015 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e98d2bd-e98b-4f73-8c3a-d3bd48bafaa1 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:05:25.557612 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302c8d9a-b726-4c9c-a861-6fc5e4972e3c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c277338a7f814cc6b01c5419775031bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e30ccf0c1204727b786876c141dbcd2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:05:25.558242 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302c8d9a-b726-4c9c-a861-6fc5e4972e3c service cinder] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:05:25.558517 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302c8d9a-b726-4c9c-a861-6fc5e4972e3c service cinder] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:05:25.558782 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302c8d9a-b726-4c9c-a861-6fc5e4972e3c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:05:25.567536 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-302c8d9a-b726-4c9c-a861-6fc5e4972e3c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:05:25.569551 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-302c8d9a-b726-4c9c-a861-6fc5e4972e3c service cinder] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:05:25.578789 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 981/1945] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 11:05:25 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 11:05:25.620587 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99de0a73-c431-434e-9f01-9c2520eb5990 None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:05:25.630251 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99de0a73-c431-434e-9f01-9c2520eb5990 None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:05:25.634813 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64125ffb-dec7-4685-84ea-a5d0f4663539 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c277338a7f814cc6b01c5419775031bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e30ccf0c1204727b786876c141dbcd2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:05:25.635461 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64125ffb-dec7-4685-84ea-a5d0f4663539 service cinder] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:05:25.635749 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64125ffb-dec7-4685-84ea-a5d0f4663539 service cinder] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:05:25.636024 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64125ffb-dec7-4685-84ea-a5d0f4663539 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:05:25.736212 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64125ffb-dec7-4685-84ea-a5d0f4663539 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:05:25.739056 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64125ffb-dec7-4685-84ea-a5d0f4663539 service cinder] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:05:25.752902 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 965/1946] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 11:05:25 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 11:05:35.574604 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:INSERT=4,keystone:SELECT=43 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 11:05:35.747685 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=51 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 11:05:41.106556 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302c8d9a-b726-4c9c-a861-6fc5e4972e3c service cinder] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:05:41.108044 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-302c8d9a-b726-4c9c-a861-6fc5e4972e3c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:05:41.191978 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302c8d9a-b726-4c9c-a861-6fc5e4972e3c service cinder] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:05:41.198315 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed89aefe-2be3-4dcf-a524-2a705558a7cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:05:41.199092 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed89aefe-2be3-4dcf-a524-2a705558a7cc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:05:41.199441 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed89aefe-2be3-4dcf-a524-2a705558a7cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:05:41.207325 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed89aefe-2be3-4dcf-a524-2a705558a7cc admin admin] PATH_INFO: `/v3/users/8d1b7fe6be55401690e910148258a90a` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:05:41.207325 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed89aefe-2be3-4dcf-a524-2a705558a7cc admin admin] RBAC: Authorizing `identity:delete_user(user_id=8d1b7fe6be55401690e910148258a90a)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:05:41.207325 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed89aefe-2be3-4dcf-a524-2a705558a7cc admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:05:41.286222 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-ed89aefe-2be3-4dcf-a524-2a705558a7cc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8d1b7fe6be55401690e910148258a90a'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:05:41.286987 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-ed89aefe-2be3-4dcf-a524-2a705558a7cc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8d1b7fe6be55401690e910148258a90a'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:05:41.295439 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-ed89aefe-2be3-4dcf-a524-2a705558a7cc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8d1b7fe6be55401690e910148258a90a'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:05:41.296172 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-ed89aefe-2be3-4dcf-a524-2a705558a7cc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8d1b7fe6be55401690e910148258a90a'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:05:41.311634 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-ed89aefe-2be3-4dcf-a524-2a705558a7cc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8d1b7fe6be55401690e910148258a90a'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:05:41.328556 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 982/1947] 10.222.0.40 () {58 vars in 1224 bytes} [Fri Aug 26 11:05:41 2022] DELETE /identity/v3/users/8d1b7fe6be55401690e910148258a90a => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 11:05:41.333051 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1de6b50-49b9-46a1-8407-00e00df17cc1 None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:05:41.333555 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1de6b50-49b9-46a1-8407-00e00df17cc1 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:05:41.333637 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1de6b50-49b9-46a1-8407-00e00df17cc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:05:41.361739 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-a1de6b50-49b9-46a1-8407-00e00df17cc1 None None] MFA Rules not processed for user `ca155a48e2024c32af3fd2e19e5fac11`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 11:05:41.363477 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-a1de6b50-49b9-46a1-8407-00e00df17cc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:05:41.444846 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 966/1948] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 11:05:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 11:05:41.454295 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed89aefe-2be3-4dcf-a524-2a705558a7cc admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:05:41.455833 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-ed89aefe-2be3-4dcf-a524-2a705558a7cc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:05:41.533693 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed89aefe-2be3-4dcf-a524-2a705558a7cc admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:05:41.539276 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6fdaef7-05c0-41fa-9e77-d9c4f4a5e5d0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91da2acd2534423091537e1ba068d2b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e30ccf0c1204727b786876c141dbcd2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:05:41.539983 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fdaef7-05c0-41fa-9e77-d9c4f4a5e5d0 service neutron] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:05:41.540880 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fdaef7-05c0-41fa-9e77-d9c4f4a5e5d0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:05:41.541234 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fdaef7-05c0-41fa-9e77-d9c4f4a5e5d0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:05:41.576041 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6fdaef7-05c0-41fa-9e77-d9c4f4a5e5d0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:05:41.577822 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6fdaef7-05c0-41fa-9e77-d9c4f4a5e5d0 service neutron] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:05:41.590484 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 983/1949] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 11:05:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 11:05:41.713298 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1de6b50-49b9-46a1-8407-00e00df17cc1 None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:05:41.715360 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-a1de6b50-49b9-46a1-8407-00e00df17cc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:05:41.729266 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1de6b50-49b9-46a1-8407-00e00df17cc1 None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:05:41.741173 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c716237-a55b-43bb-9ec0-80b397c10447 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:05:41.742444 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c716237-a55b-43bb-9ec0-80b397c10447 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:05:41.742851 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c716237-a55b-43bb-9ec0-80b397c10447 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:05:41.743172 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c716237-a55b-43bb-9ec0-80b397c10447 admin admin] PATH_INFO: `/v3/users/7bfa3999c64e4e3b9e6fd420e7e6ed9a` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:05:41.744711 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c716237-a55b-43bb-9ec0-80b397c10447 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7bfa3999c64e4e3b9e6fd420e7e6ed9a)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:05:41.746667 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c716237-a55b-43bb-9ec0-80b397c10447 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:05:41.813742 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-0c716237-a55b-43bb-9ec0-80b397c10447 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7bfa3999c64e4e3b9e6fd420e7e6ed9a'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:05:41.823145 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-0c716237-a55b-43bb-9ec0-80b397c10447 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7bfa3999c64e4e3b9e6fd420e7e6ed9a'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:05:41.823941 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-0c716237-a55b-43bb-9ec0-80b397c10447 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7bfa3999c64e4e3b9e6fd420e7e6ed9a'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:05:41.840897 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-0c716237-a55b-43bb-9ec0-80b397c10447 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7bfa3999c64e4e3b9e6fd420e7e6ed9a'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:05:41.841710 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-0c716237-a55b-43bb-9ec0-80b397c10447 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7bfa3999c64e4e3b9e6fd420e7e6ed9a'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:05:41.870061 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 967/1950] 10.222.0.40 () {58 vars in 1224 bytes} [Fri Aug 26 11:05:41 2022] DELETE /identity/v3/users/7bfa3999c64e4e3b9e6fd420e7e6ed9a => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 11:05:41.872999 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6525ab-1317-4703-b28e-0890b7f53d72 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:05:41.873413 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6525ab-1317-4703-b28e-0890b7f53d72 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:05:41.873783 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6525ab-1317-4703-b28e-0890b7f53d72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:05:41.897366 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-7d6525ab-1317-4703-b28e-0890b7f53d72 None None] MFA Rules not processed for user `ca155a48e2024c32af3fd2e19e5fac11`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 11:05:41.899052 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-7d6525ab-1317-4703-b28e-0890b7f53d72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:05:41.920759 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 11:05:41.980143 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 984/1951] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 11:05:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 11:05:41.987178 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c716237-a55b-43bb-9ec0-80b397c10447 admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:05:41.989523 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-0c716237-a55b-43bb-9ec0-80b397c10447 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:05:41.994984 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6525ab-1317-4703-b28e-0890b7f53d72 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:05:41.996559 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-7d6525ab-1317-4703-b28e-0890b7f53d72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:05:42.034729 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c716237-a55b-43bb-9ec0-80b397c10447 admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:05:42.046756 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61248964-2046-418b-86f6-65be9e2186a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:05:42.047657 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61248964-2046-418b-86f6-65be9e2186a2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:05:42.048225 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61248964-2046-418b-86f6-65be9e2186a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:05:42.049744 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61248964-2046-418b-86f6-65be9e2186a2 admin admin] PATH_INFO: `/v3/projects/08b7d78012a947cb9f1b78933fa94b83` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:05:42.051240 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61248964-2046-418b-86f6-65be9e2186a2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=08b7d78012a947cb9f1b78933fa94b83)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:05:42.054502 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61248964-2046-418b-86f6-65be9e2186a2 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:05:42.123192 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6525ab-1317-4703-b28e-0890b7f53d72 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:05:42.135327 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-61248964-2046-418b-86f6-65be9e2186a2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '08b7d78012a947cb9f1b78933fa94b83', 'request_id': 'req-61248964-2046-418b-86f6-65be9e2186a2'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:05:42.142719 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-61248964-2046-418b-86f6-65be9e2186a2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '08b7d78012a947cb9f1b78933fa94b83', 'request_id': 'req-61248964-2046-418b-86f6-65be9e2186a2'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:05:42.150514 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-61248964-2046-418b-86f6-65be9e2186a2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '08b7d78012a947cb9f1b78933fa94b83', 'request_id': 'req-61248964-2046-418b-86f6-65be9e2186a2'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:05:42.156000 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 11:05:42.167882 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-61248964-2046-418b-86f6-65be9e2186a2 admin admin] The token cache is being invalidate because project 08b7d78012a947cb9f1b78933fa94b83 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81423) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 11:05:42.171574 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-61248964-2046-418b-86f6-65be9e2186a2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:05:42.172520 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-61248964-2046-418b-86f6-65be9e2186a2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:05:42.174401 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 968/1952] 10.222.0.40 () {58 vars in 1233 bytes} [Fri Aug 26 11:05:41 2022] DELETE /identity/v3/projects/08b7d78012a947cb9f1b78933fa94b83 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 11:05:42.179464 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61248964-2046-418b-86f6-65be9e2186a2 admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:05:42.182832 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-61248964-2046-418b-86f6-65be9e2186a2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:05:42.205182 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f9946d-63cf-4543-9019-22378a408fe0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91da2acd2534423091537e1ba068d2b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e30ccf0c1204727b786876c141dbcd2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:05:42.206191 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f9946d-63cf-4543-9019-22378a408fe0 service neutron] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:05:42.207180 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f9946d-63cf-4543-9019-22378a408fe0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:05:42.220547 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f9946d-63cf-4543-9019-22378a408fe0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:05:42.223256 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-03f9946d-63cf-4543-9019-22378a408fe0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:05:42.285832 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61248964-2046-418b-86f6-65be9e2186a2 admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:05:42.292816 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d9ae9b-549d-4efc-9983-12917ccdcbec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:05:42.293870 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d9ae9b-549d-4efc-9983-12917ccdcbec admin admin] REQUEST_METHOD: `DELETE` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:05:42.294515 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d9ae9b-549d-4efc-9983-12917ccdcbec admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:05:42.295046 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d9ae9b-549d-4efc-9983-12917ccdcbec admin admin] PATH_INFO: `/v3/users/b99bd5e828a347749ecb96b9c17004ce` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:05:42.297237 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d9ae9b-549d-4efc-9983-12917ccdcbec admin admin] RBAC: Authorizing `identity:delete_user(user_id=b99bd5e828a347749ecb96b9c17004ce)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:05:42.299823 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d9ae9b-549d-4efc-9983-12917ccdcbec admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:05:42.318202 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f9946d-63cf-4543-9019-22378a408fe0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:05:42.320393 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f9946d-63cf-4543-9019-22378a408fe0 service neutron] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:05:42.335467 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 985/1953] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 11:05:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 26 11:05:42.380182 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-94d9ae9b-549d-4efc-9983-12917ccdcbec admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b99bd5e828a347749ecb96b9c17004ce'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:05:42.390547 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-94d9ae9b-549d-4efc-9983-12917ccdcbec admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b99bd5e828a347749ecb96b9c17004ce'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:05:42.404273 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-94d9ae9b-549d-4efc-9983-12917ccdcbec admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b99bd5e828a347749ecb96b9c17004ce'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:05:42.423459 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-94d9ae9b-549d-4efc-9983-12917ccdcbec admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b99bd5e828a347749ecb96b9c17004ce'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:05:42.424235 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-94d9ae9b-549d-4efc-9983-12917ccdcbec admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b99bd5e828a347749ecb96b9c17004ce'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:05:42.442500 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 969/1954] 10.222.0.40 () {58 vars in 1224 bytes} [Fri Aug 26 11:05:42 2022] DELETE /identity/v3/users/b99bd5e828a347749ecb96b9c17004ce => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 11:05:42.915036 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f9946d-63cf-4543-9019-22378a408fe0 service neutron] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:05:42.917266 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-03f9946d-63cf-4543-9019-22378a408fe0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:05:42.942916 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f9946d-63cf-4543-9019-22378a408fe0 service neutron] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:05:43.016009 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 11:05:43.016843 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d9ae9b-549d-4efc-9983-12917ccdcbec admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:05:43.018412 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-94d9ae9b-549d-4efc-9983-12917ccdcbec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:05:43.116353 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a80aa71-8264-4f1b-9463-203122af9526 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:05:43.125141 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a80aa71-8264-4f1b-9463-203122af9526 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:05:43.125780 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a80aa71-8264-4f1b-9463-203122af9526 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:05:43.125780 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a80aa71-8264-4f1b-9463-203122af9526 admin admin] PATH_INFO: `/v3/projects/0dfb6a01448a4a128b1f6fe65532f9e8` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:05:43.127013 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a80aa71-8264-4f1b-9463-203122af9526 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0dfb6a01448a4a128b1f6fe65532f9e8)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:05:43.128975 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a80aa71-8264-4f1b-9463-203122af9526 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:05:43.204959 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d9ae9b-549d-4efc-9983-12917ccdcbec admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:05:43.238917 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08eca4d6-e560-4107-ad92-25358c158cf3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:05:43.244118 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08eca4d6-e560-4107-ad92-25358c158cf3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:05:43.244354 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08eca4d6-e560-4107-ad92-25358c158cf3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:05:43.244632 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08eca4d6-e560-4107-ad92-25358c158cf3 admin admin] PATH_INFO: `/v3/projects/1641c7f2ffe04631945584457d74c441` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:05:43.248227 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-5a80aa71-8264-4f1b-9463-203122af9526 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0dfb6a01448a4a128b1f6fe65532f9e8', 'request_id': 'req-5a80aa71-8264-4f1b-9463-203122af9526'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:05:43.248790 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08eca4d6-e560-4107-ad92-25358c158cf3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1641c7f2ffe04631945584457d74c441)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:05:43.253293 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08eca4d6-e560-4107-ad92-25358c158cf3 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:05:43.254693 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-5a80aa71-8264-4f1b-9463-203122af9526 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0dfb6a01448a4a128b1f6fe65532f9e8', 'request_id': 'req-5a80aa71-8264-4f1b-9463-203122af9526'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:05:43.261172 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-5a80aa71-8264-4f1b-9463-203122af9526 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0dfb6a01448a4a128b1f6fe65532f9e8', 'request_id': 'req-5a80aa71-8264-4f1b-9463-203122af9526'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:05:43.278506 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-5a80aa71-8264-4f1b-9463-203122af9526 admin admin] The token cache is being invalidate because project 0dfb6a01448a4a128b1f6fe65532f9e8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81424) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 11:05:43.279072 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-5a80aa71-8264-4f1b-9463-203122af9526 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:05:43.279718 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-5a80aa71-8264-4f1b-9463-203122af9526 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:05:43.282031 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 986/1955] 10.222.0.40 () {58 vars in 1233 bytes} [Fri Aug 26 11:05:42 2022] DELETE /identity/v3/projects/0dfb6a01448a4a128b1f6fe65532f9e8 => generated 0 bytes in 369 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 11:05:43.289762 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 11:05:43.290698 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a80aa71-8264-4f1b-9463-203122af9526 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:05:43.293003 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-5a80aa71-8264-4f1b-9463-203122af9526 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:05:43.336168 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-08eca4d6-e560-4107-ad92-25358c158cf3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1641c7f2ffe04631945584457d74c441', 'request_id': 'req-08eca4d6-e560-4107-ad92-25358c158cf3'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:05:43.343085 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-08eca4d6-e560-4107-ad92-25358c158cf3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1641c7f2ffe04631945584457d74c441', 'request_id': 'req-08eca4d6-e560-4107-ad92-25358c158cf3'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:05:43.349189 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-08eca4d6-e560-4107-ad92-25358c158cf3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1641c7f2ffe04631945584457d74c441', 'request_id': 'req-08eca4d6-e560-4107-ad92-25358c158cf3'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:05:43.368498 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-08eca4d6-e560-4107-ad92-25358c158cf3 admin admin] The token cache is being invalidate because project 1641c7f2ffe04631945584457d74c441 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81423) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 11:05:43.369897 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-08eca4d6-e560-4107-ad92-25358c158cf3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:05:43.370772 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-08eca4d6-e560-4107-ad92-25358c158cf3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:05:43.375286 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 970/1956] 10.222.0.40 () {58 vars in 1233 bytes} [Fri Aug 26 11:05:43 2022] DELETE /identity/v3/projects/1641c7f2ffe04631945584457d74c441 => generated 0 bytes in 359 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 26 11:05:43.380963 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2424edf7-0e13-4ae3-bc56-0c242cde40f1 None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:05:43.382388 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2424edf7-0e13-4ae3-bc56-0c242cde40f1 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:05:43.382783 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2424edf7-0e13-4ae3-bc56-0c242cde40f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:05:43.399074 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a80aa71-8264-4f1b-9463-203122af9526 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:05:43.405757 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-2424edf7-0e13-4ae3-bc56-0c242cde40f1 None None] MFA Rules not processed for user `ca155a48e2024c32af3fd2e19e5fac11`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 11:05:43.407057 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-2424edf7-0e13-4ae3-bc56-0c242cde40f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:05:43.546423 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 971/1957] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 11:05:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 11:05:43.549858 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5eab6f9-ee40-4fd2-83de-2ebf1ad7cda1 None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:05:43.550174 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5eab6f9-ee40-4fd2-83de-2ebf1ad7cda1 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:05:43.550452 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5eab6f9-ee40-4fd2-83de-2ebf1ad7cda1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:05:43.588919 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-d5eab6f9-ee40-4fd2-83de-2ebf1ad7cda1 None None] MFA Rules not processed for user `ca155a48e2024c32af3fd2e19e5fac11`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 11:05:43.593117 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-d5eab6f9-ee40-4fd2-83de-2ebf1ad7cda1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:05:43.605867 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c7421f-9124-4db3-86e7-1e77f49b6a8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:05:43.607593 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c7421f-9124-4db3-86e7-1e77f49b6a8f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:05:43.608055 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c7421f-9124-4db3-86e7-1e77f49b6a8f admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:05:43.608331 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c7421f-9124-4db3-86e7-1e77f49b6a8f admin admin] PATH_INFO: `/v3/users/88ecd0cc5f2844ccadfe386e769f14f1` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:05:43.610532 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c7421f-9124-4db3-86e7-1e77f49b6a8f admin admin] RBAC: Authorizing `identity:delete_user(user_id=88ecd0cc5f2844ccadfe386e769f14f1)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:05:43.616729 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c7421f-9124-4db3-86e7-1e77f49b6a8f admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:05:43.621027 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 972/1958] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 11:05:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 71 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 11:05:43.622939 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5eab6f9-ee40-4fd2-83de-2ebf1ad7cda1 None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:05:43.632165 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 11:05:43.640711 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5eab6f9-ee40-4fd2-83de-2ebf1ad7cda1 None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:05:43.649489 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a98dfe1-3880-40c2-9ff5-ff698ced9829 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:05:43.650455 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a98dfe1-3880-40c2-9ff5-ff698ced9829 admin admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:05:43.650737 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a98dfe1-3880-40c2-9ff5-ff698ced9829 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:05:43.651213 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a98dfe1-3880-40c2-9ff5-ff698ced9829 admin admin] PATH_INFO: `/v3/domains` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:05:43.652172 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a98dfe1-3880-40c2-9ff5-ff698ced9829 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 11:05:43.652618 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a98dfe1-3880-40c2-9ff5-ff698ced9829 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:05:43.655308 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a98dfe1-3880-40c2-9ff5-ff698ced9829 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:05:43.673226 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 973/1959] 10.222.0.40 () {58 vars in 1153 bytes} [Fri Aug 26 11:05:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 11:05:43.674256 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a98dfe1-3880-40c2-9ff5-ff698ced9829 admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:05:43.687168 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a98dfe1-3880-40c2-9ff5-ff698ced9829 admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:05:43.694393 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181248fd-ae59-41d7-b4b3-c4ecf04c1280 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:05:43.695043 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181248fd-ae59-41d7-b4b3-c4ecf04c1280 admin admin] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:05:43.695380 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181248fd-ae59-41d7-b4b3-c4ecf04c1280 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:05:43.695659 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181248fd-ae59-41d7-b4b3-c4ecf04c1280 admin admin] PATH_INFO: `/v3/projects` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:05:43.696346 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-181248fd-ae59-41d7-b4b3-c4ecf04c1280 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:05:43.698338 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-181248fd-ae59-41d7-b4b3-c4ecf04c1280 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:05:43.712451 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-c2c7421f-9124-4db3-86e7-1e77f49b6a8f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '88ecd0cc5f2844ccadfe386e769f14f1'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:05:43.713291 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-c2c7421f-9124-4db3-86e7-1e77f49b6a8f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '88ecd0cc5f2844ccadfe386e769f14f1'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:05:43.725848 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-c2c7421f-9124-4db3-86e7-1e77f49b6a8f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '88ecd0cc5f2844ccadfe386e769f14f1'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:05:43.729316 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-c2c7421f-9124-4db3-86e7-1e77f49b6a8f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '88ecd0cc5f2844ccadfe386e769f14f1'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:05:43.745910 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-c2c7421f-9124-4db3-86e7-1e77f49b6a8f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '88ecd0cc5f2844ccadfe386e769f14f1'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:05:43.747281 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 974/1960] 10.222.0.40 () {60 vars in 1153 bytes} [Fri Aug 26 11:05:43 2022] POST /identity/v3/projects => generated 383 bytes in 74 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 11:05:43.751722 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181248fd-ae59-41d7-b4b3-c4ecf04c1280 admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:05:43.753832 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-181248fd-ae59-41d7-b4b3-c4ecf04c1280 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:05:43.769271 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 987/1961] 10.222.0.40 () {58 vars in 1224 bytes} [Fri Aug 26 11:05:43 2022] DELETE /identity/v3/users/88ecd0cc5f2844ccadfe386e769f14f1 => generated 0 bytes in 483 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 11:05:43.843797 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181248fd-ae59-41d7-b4b3-c4ecf04c1280 admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:05:43.858001 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 11:05:43.958206 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b6483c-9949-452b-9aa5-d41a24eda059 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:05:43.959700 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b6483c-9949-452b-9aa5-d41a24eda059 admin admin] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:05:43.959985 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b6483c-9949-452b-9aa5-d41a24eda059 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:05:43.960355 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b6483c-9949-452b-9aa5-d41a24eda059 admin admin] PATH_INFO: `/v3/users` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:05:43.961018 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b6483c-9949-452b-9aa5-d41a24eda059 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:05:43.963443 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b6483c-9949-452b-9aa5-d41a24eda059 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:05:44.013489 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 975/1962] 10.222.0.40 () {60 vars in 1144 bytes} [Fri Aug 26 11:05:43 2022] POST /identity/v3/users => generated 292 bytes in 259 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 26 11:05:44.014957 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c7421f-9124-4db3-86e7-1e77f49b6a8f admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:05:44.028989 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c7421f-9124-4db3-86e7-1e77f49b6a8f admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:05:44.039773 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50d77cb-5673-4940-8202-3ac930a6f8ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:05:44.040391 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50d77cb-5673-4940-8202-3ac930a6f8ec admin admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:05:44.040721 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50d77cb-5673-4940-8202-3ac930a6f8ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:05:44.040998 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50d77cb-5673-4940-8202-3ac930a6f8ec admin admin] PATH_INFO: `/v3/roles` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:05:44.041648 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b50d77cb-5673-4940-8202-3ac930a6f8ec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:05:44.045678 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b50d77cb-5673-4940-8202-3ac930a6f8ec admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:05:44.069913 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 988/1963] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 26 11:05:44 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 11:05:44.073110 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b6483c-9949-452b-9aa5-d41a24eda059 admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:05:44.085843 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b6483c-9949-452b-9aa5-d41a24eda059 admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:05:44.092064 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790b9e05-2e1c-4ca8-9495-2cf2bfb8ede6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:05:44.092979 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790b9e05-2e1c-4ca8-9495-2cf2bfb8ede6 admin admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:05:44.093350 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790b9e05-2e1c-4ca8-9495-2cf2bfb8ede6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:05:44.093686 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790b9e05-2e1c-4ca8-9495-2cf2bfb8ede6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:05:44.094792 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-790b9e05-2e1c-4ca8-9495-2cf2bfb8ede6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:05:44.096989 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-790b9e05-2e1c-4ca8-9495-2cf2bfb8ede6 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:05:44.110123 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 976/1964] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 26 11:05:44 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 26 11:05:44.114578 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50d77cb-5673-4940-8202-3ac930a6f8ec admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:05:44.127589 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50d77cb-5673-4940-8202-3ac930a6f8ec admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:05:44.135314 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef2d845-9993-401c-a9b3-49ba822780c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:05:44.136212 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef2d845-9993-401c-a9b3-49ba822780c2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:05:44.136581 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef2d845-9993-401c-a9b3-49ba822780c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:05:44.136925 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef2d845-9993-401c-a9b3-49ba822780c2 admin admin] PATH_INFO: `/v3/projects/950827994dd14b86b76a399de03b563b/users/f5e361c7ef1745ae9dc2b499a591206b/roles/23e174ce743b46eabe618c212691da55` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:05:44.159642 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef2d845-9993-401c-a9b3-49ba822780c2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=950827994dd14b86b76a399de03b563b, user_id=f5e361c7ef1745ae9dc2b499a591206b, role_id=23e174ce743b46eabe618c212691da55)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:05:44.163106 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef2d845-9993-401c-a9b3-49ba822780c2 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:05:44.182757 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790b9e05-2e1c-4ca8-9495-2cf2bfb8ede6 admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:05:44.185279 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 989/1965] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 26 11:05:44 2022] PUT /identity/v3/projects/950827994dd14b86b76a399de03b563b/users/f5e361c7ef1745ae9dc2b499a591206b/roles/23e174ce743b46eabe618c212691da55 => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 26 11:05:44.191985 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-790b9e05-2e1c-4ca8-9495-2cf2bfb8ede6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:05:44.197356 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a80f413-0690-4362-a663-e3ff2d574e00 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:05:44.197498 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a80f413-0690-4362-a663-e3ff2d574e00 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:05:44.197875 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a80f413-0690-4362-a663-e3ff2d574e00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:05:44.233418 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-4a80f413-0690-4362-a663-e3ff2d574e00 None None] MFA Rules not processed for user `f5e361c7ef1745ae9dc2b499a591206b`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 11:05:44.235378 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-4a80f413-0690-4362-a663-e3ff2d574e00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:05:44.254183 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790b9e05-2e1c-4ca8-9495-2cf2bfb8ede6 admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:05:44.348247 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 990/1966] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 11:05:44 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 11:05:44.349139 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7f1050-a0ba-42a1-9b50-3a4f616ac6ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:05:44.349528 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a80f413-0690-4362-a663-e3ff2d574e00 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:05:44.350020 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7f1050-a0ba-42a1-9b50-3a4f616ac6ac admin admin] REQUEST_METHOD: `DELETE` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:05:44.350306 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7f1050-a0ba-42a1-9b50-3a4f616ac6ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:05:44.350583 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7f1050-a0ba-42a1-9b50-3a4f616ac6ac admin admin] PATH_INFO: `/v3/projects/f37f73dcfccf49219b51fc4072114736` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:05:44.351762 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e7f1050-a0ba-42a1-9b50-3a4f616ac6ac admin admin] RBAC: Authorizing `identity:delete_project(project_id=f37f73dcfccf49219b51fc4072114736)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:05:44.353684 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e7f1050-a0ba-42a1-9b50-3a4f616ac6ac admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:05:44.367854 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a80f413-0690-4362-a663-e3ff2d574e00 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:05:44.376553 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b61902d-efbf-4fb1-a2e6-cbcc91ad2c71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:05:44.377089 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b61902d-efbf-4fb1-a2e6-cbcc91ad2c71 admin admin] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:05:44.377361 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b61902d-efbf-4fb1-a2e6-cbcc91ad2c71 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:05:44.377660 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b61902d-efbf-4fb1-a2e6-cbcc91ad2c71 admin admin] PATH_INFO: `/v3/projects` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:05:44.379422 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b61902d-efbf-4fb1-a2e6-cbcc91ad2c71 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:05:44.382519 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b61902d-efbf-4fb1-a2e6-cbcc91ad2c71 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:05:44.421970 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 991/1967] 10.222.0.40 () {60 vars in 1153 bytes} [Fri Aug 26 11:05:44 2022] POST /identity/v3/projects => generated 381 bytes in 73 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 11:05:44.430885 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b61902d-efbf-4fb1-a2e6-cbcc91ad2c71 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:05:44.454861 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-8e7f1050-a0ba-42a1-9b50-3a4f616ac6ac admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f37f73dcfccf49219b51fc4072114736', 'request_id': 'req-8e7f1050-a0ba-42a1-9b50-3a4f616ac6ac'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:05:44.462486 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-8e7f1050-a0ba-42a1-9b50-3a4f616ac6ac admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f37f73dcfccf49219b51fc4072114736', 'request_id': 'req-8e7f1050-a0ba-42a1-9b50-3a4f616ac6ac'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:05:44.469330 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-8e7f1050-a0ba-42a1-9b50-3a4f616ac6ac admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f37f73dcfccf49219b51fc4072114736', 'request_id': 'req-8e7f1050-a0ba-42a1-9b50-3a4f616ac6ac'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:05:44.478796 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=87,keystone:DELETE=10,keystone:INSERT=3 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 11:05:44.486504 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-8e7f1050-a0ba-42a1-9b50-3a4f616ac6ac admin admin] The token cache is being invalidate because project f37f73dcfccf49219b51fc4072114736 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81423) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 11:05:44.486835 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-8e7f1050-a0ba-42a1-9b50-3a4f616ac6ac admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:05:44.487544 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-8e7f1050-a0ba-42a1-9b50-3a4f616ac6ac admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:05:44.493418 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 977/1968] 10.222.0.40 () {58 vars in 1233 bytes} [Fri Aug 26 11:05:44 2022] DELETE /identity/v3/projects/f37f73dcfccf49219b51fc4072114736 => generated 0 bytes in 310 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 11:05:44.498295 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7f1050-a0ba-42a1-9b50-3a4f616ac6ac admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:05:44.500138 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-8e7f1050-a0ba-42a1-9b50-3a4f616ac6ac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:05:44.531330 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 11:05:44.582221 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b61902d-efbf-4fb1-a2e6-cbcc91ad2c71 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:05:44.599120 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7f1050-a0ba-42a1-9b50-3a4f616ac6ac admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:05:44.606915 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc0c514-4efa-493a-99fc-3f31e67ddda8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:05:44.608051 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc0c514-4efa-493a-99fc-3f31e67ddda8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:05:44.609201 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc0c514-4efa-493a-99fc-3f31e67ddda8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:05:44.609703 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc0c514-4efa-493a-99fc-3f31e67ddda8 admin admin] PATH_INFO: `/v3/users/d96ba4e7e8954058bdbaa6f64673ac4c` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:05:44.611368 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc0c514-4efa-493a-99fc-3f31e67ddda8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d96ba4e7e8954058bdbaa6f64673ac4c)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:05:44.614331 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc0c514-4efa-493a-99fc-3f31e67ddda8 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:05:44.668901 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b77902d-008b-4e12-a7f7-d0f2b1523adc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:05:44.669638 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b77902d-008b-4e12-a7f7-d0f2b1523adc admin admin] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:05:44.669871 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b77902d-008b-4e12-a7f7-d0f2b1523adc admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:05:44.670174 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b77902d-008b-4e12-a7f7-d0f2b1523adc admin admin] PATH_INFO: `/v3/users` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:05:44.670889 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b77902d-008b-4e12-a7f7-d0f2b1523adc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:05:44.672741 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b77902d-008b-4e12-a7f7-d0f2b1523adc admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:05:44.696056 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-cfc0c514-4efa-493a-99fc-3f31e67ddda8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd96ba4e7e8954058bdbaa6f64673ac4c'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:05:44.704314 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-cfc0c514-4efa-493a-99fc-3f31e67ddda8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd96ba4e7e8954058bdbaa6f64673ac4c'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:05:44.704753 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 992/1969] 10.222.0.40 () {60 vars in 1144 bytes} [Fri Aug 26 11:05:44 2022] POST /identity/v3/users => generated 297 bytes in 278 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 11:05:44.708531 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b77902d-008b-4e12-a7f7-d0f2b1523adc admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:05:44.708833 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-cfc0c514-4efa-493a-99fc-3f31e67ddda8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd96ba4e7e8954058bdbaa6f64673ac4c'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:05:44.710744 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-1b77902d-008b-4e12-a7f7-d0f2b1523adc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:05:44.727189 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-cfc0c514-4efa-493a-99fc-3f31e67ddda8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd96ba4e7e8954058bdbaa6f64673ac4c'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:05:44.728320 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-cfc0c514-4efa-493a-99fc-3f31e67ddda8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd96ba4e7e8954058bdbaa6f64673ac4c'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:05:44.732434 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b77902d-008b-4e12-a7f7-d0f2b1523adc admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:05:44.738969 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522e345c-940a-45eb-abed-c8a5d43f68a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:05:44.739667 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522e345c-940a-45eb-abed-c8a5d43f68a5 admin admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:05:44.740629 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522e345c-940a-45eb-abed-c8a5d43f68a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:05:44.740976 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522e345c-940a-45eb-abed-c8a5d43f68a5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:05:44.741799 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-522e345c-940a-45eb-abed-c8a5d43f68a5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:05:44.743793 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-522e345c-940a-45eb-abed-c8a5d43f68a5 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:05:44.756942 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 993/1970] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 26 11:05:44 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 26 11:05:44.763030 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522e345c-940a-45eb-abed-c8a5d43f68a5 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:05:44.767376 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 978/1971] 10.222.0.40 () {58 vars in 1224 bytes} [Fri Aug 26 11:05:44 2022] DELETE /identity/v3/users/d96ba4e7e8954058bdbaa6f64673ac4c => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 11:05:44.770375 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-522e345c-940a-45eb-abed-c8a5d43f68a5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:05:44.798551 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522e345c-940a-45eb-abed-c8a5d43f68a5 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:05:44.940299 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e22b22-4166-42f0-bd47-6d4affa13255 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:05:44.941067 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e22b22-4166-42f0-bd47-6d4affa13255 admin admin] REQUEST_METHOD: `PUT` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:05:44.941356 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e22b22-4166-42f0-bd47-6d4affa13255 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:05:44.941745 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e22b22-4166-42f0-bd47-6d4affa13255 admin admin] PATH_INFO: `/v3/projects/0e6d7f8fe42d4678a3cfffdb631cac4d/users/7411c7058ca74601a2806675ab2ae179/roles/1f97e4602e2e4d6498913a81f9ae8d6a` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:05:44.960606 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e22b22-4166-42f0-bd47-6d4affa13255 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0e6d7f8fe42d4678a3cfffdb631cac4d, user_id=7411c7058ca74601a2806675ab2ae179, role_id=1f97e4602e2e4d6498913a81f9ae8d6a)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:05:44.964373 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e22b22-4166-42f0-bd47-6d4affa13255 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:05:44.985593 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 994/1972] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 26 11:05:44 2022] PUT /identity/v3/projects/0e6d7f8fe42d4678a3cfffdb631cac4d/users/7411c7058ca74601a2806675ab2ae179/roles/1f97e4602e2e4d6498913a81f9ae8d6a => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 11:05:44.991303 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc0c514-4efa-493a-99fc-3f31e67ddda8 admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:05:45.009838 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc0c514-4efa-493a-99fc-3f31e67ddda8 admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:05:45.097061 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de0f786-5d4e-4a1b-bc69-d15af6fa1eda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:05:45.097646 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de0f786-5d4e-4a1b-bc69-d15af6fa1eda admin admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:05:45.097875 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de0f786-5d4e-4a1b-bc69-d15af6fa1eda admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:05:45.098182 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de0f786-5d4e-4a1b-bc69-d15af6fa1eda admin admin] PATH_INFO: `/v3/roles` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:05:45.100941 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de0f786-5d4e-4a1b-bc69-d15af6fa1eda admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:05:45.102975 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de0f786-5d4e-4a1b-bc69-d15af6fa1eda admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:05:45.118021 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 979/1973] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 26 11:05:44 2022] GET /identity/v3/roles => generated 1420 bytes in 124 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 11:05:45.119606 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e22b22-4166-42f0-bd47-6d4affa13255 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:05:45.132830 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e22b22-4166-42f0-bd47-6d4affa13255 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:05:45.139921 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0624c36-36d7-468c-a114-3676281fdd6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:05:45.140610 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0624c36-36d7-468c-a114-3676281fdd6d admin admin] REQUEST_METHOD: `PUT` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:05:45.140955 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0624c36-36d7-468c-a114-3676281fdd6d admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:05:45.141180 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0624c36-36d7-468c-a114-3676281fdd6d admin admin] PATH_INFO: `/v3/projects/0e6d7f8fe42d4678a3cfffdb631cac4d/users/7411c7058ca74601a2806675ab2ae179/roles/1f97e4602e2e4d6498913a81f9ae8d6a` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:05:45.143417 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0624c36-36d7-468c-a114-3676281fdd6d admin admin] RBAC: Authorizing `identity:create_grant(project_id=0e6d7f8fe42d4678a3cfffdb631cac4d, user_id=7411c7058ca74601a2806675ab2ae179, role_id=1f97e4602e2e4d6498913a81f9ae8d6a)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:05:45.146919 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0624c36-36d7-468c-a114-3676281fdd6d admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:05:45.159077 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 995/1974] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 26 11:05:45 2022] PUT /identity/v3/projects/0e6d7f8fe42d4678a3cfffdb631cac4d/users/7411c7058ca74601a2806675ab2ae179/roles/1f97e4602e2e4d6498913a81f9ae8d6a => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 11:05:45.172903 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02b65c6-f867-4335-971f-542517816bbb None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:05:45.173188 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02b65c6-f867-4335-971f-542517816bbb None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:05:45.173458 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02b65c6-f867-4335-971f-542517816bbb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:05:45.202539 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0624c36-36d7-468c-a114-3676281fdd6d admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:05:45.205284 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-d0624c36-36d7-468c-a114-3676281fdd6d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:05:45.207147 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-c02b65c6-f867-4335-971f-542517816bbb None None] MFA Rules not processed for user `7411c7058ca74601a2806675ab2ae179`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 11:05:45.209166 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-c02b65c6-f867-4335-971f-542517816bbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:05:45.228131 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 11:05:45.259818 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 11:05:45.306635 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0624c36-36d7-468c-a114-3676281fdd6d admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:05:45.311946 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af6fb6b-24ee-4f48-9586-b5b5c89c4d09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:05:45.312623 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af6fb6b-24ee-4f48-9586-b5b5c89c4d09 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:05:45.312906 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af6fb6b-24ee-4f48-9586-b5b5c89c4d09 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:05:45.313185 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af6fb6b-24ee-4f48-9586-b5b5c89c4d09 admin admin] PATH_INFO: `/v3/projects/26fb5b128c6b48fd894424a4385f2e8f` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:05:45.314505 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af6fb6b-24ee-4f48-9586-b5b5c89c4d09 admin admin] RBAC: Authorizing `identity:delete_project(project_id=26fb5b128c6b48fd894424a4385f2e8f)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:05:45.316421 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af6fb6b-24ee-4f48-9586-b5b5c89c4d09 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:05:45.324606 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 980/1975] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 11:05:45 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 11:05:45.336909 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02b65c6-f867-4335-971f-542517816bbb None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:05:45.338864 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-c02b65c6-f867-4335-971f-542517816bbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:05:45.407845 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-5af6fb6b-24ee-4f48-9586-b5b5c89c4d09 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '26fb5b128c6b48fd894424a4385f2e8f', 'request_id': 'req-5af6fb6b-24ee-4f48-9586-b5b5c89c4d09'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:05:45.418578 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-5af6fb6b-24ee-4f48-9586-b5b5c89c4d09 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '26fb5b128c6b48fd894424a4385f2e8f', 'request_id': 'req-5af6fb6b-24ee-4f48-9586-b5b5c89c4d09'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:05:45.424497 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02b65c6-f867-4335-971f-542517816bbb None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:05:45.424630 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-5af6fb6b-24ee-4f48-9586-b5b5c89c4d09 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '26fb5b128c6b48fd894424a4385f2e8f', 'request_id': 'req-5af6fb6b-24ee-4f48-9586-b5b5c89c4d09'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:05:45.439207 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-5af6fb6b-24ee-4f48-9586-b5b5c89c4d09 admin admin] The token cache is being invalidate because project 26fb5b128c6b48fd894424a4385f2e8f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81424) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 11:05:45.439585 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-5af6fb6b-24ee-4f48-9586-b5b5c89c4d09 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:05:45.440411 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-5af6fb6b-24ee-4f48-9586-b5b5c89c4d09 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:05:45.442203 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 996/1976] 10.222.0.40 () {58 vars in 1233 bytes} [Fri Aug 26 11:05:45 2022] DELETE /identity/v3/projects/26fb5b128c6b48fd894424a4385f2e8f => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 26 11:05:45.448121 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7c890e-b419-4e6a-979b-ddcdfbdbde75 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:05:45.448287 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7c890e-b419-4e6a-979b-ddcdfbdbde75 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:05:45.448464 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7c890e-b419-4e6a-979b-ddcdfbdbde75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:05:45.470894 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-de7c890e-b419-4e6a-979b-ddcdfbdbde75 None None] MFA Rules not processed for user `ca155a48e2024c32af3fd2e19e5fac11`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 11:05:45.472106 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-de7c890e-b419-4e6a-979b-ddcdfbdbde75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:05:45.483292 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c044a3c-7f05-41cd-8e8d-72c3d83fbb51 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c277338a7f814cc6b01c5419775031bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e30ccf0c1204727b786876c141dbcd2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:05:45.483907 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c044a3c-7f05-41cd-8e8d-72c3d83fbb51 service cinder] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:05:45.484186 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c044a3c-7f05-41cd-8e8d-72c3d83fbb51 service cinder] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:05:45.484462 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c044a3c-7f05-41cd-8e8d-72c3d83fbb51 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:05:45.486300 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-0c044a3c-7f05-41cd-8e8d-72c3d83fbb51 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:05:45.539495 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 997/1977] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 11:05:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 11:05:45.548293 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13905c64-d3be-44b8-89da-5d756496b47f None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:05:45.548822 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13905c64-d3be-44b8-89da-5d756496b47f None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:05:45.549092 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13905c64-d3be-44b8-89da-5d756496b47f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:05:45.558754 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c044a3c-7f05-41cd-8e8d-72c3d83fbb51 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:05:45.558935 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c044a3c-7f05-41cd-8e8d-72c3d83fbb51 service cinder] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:05:45.573336 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 981/1978] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 11:05:45 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 237 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 26 11:05:45.574208 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-13905c64-d3be-44b8-89da-5d756496b47f None None] MFA Rules not processed for user `ca155a48e2024c32af3fd2e19e5fac11`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 11:05:45.575622 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-13905c64-d3be-44b8-89da-5d756496b47f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:05:45.584371 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 998/1979] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 11:05:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 11:05:45.588583 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c044a3c-7f05-41cd-8e8d-72c3d83fbb51 service cinder] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:05:45.605860 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c044a3c-7f05-41cd-8e8d-72c3d83fbb51 service cinder] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:05:45.611251 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a135e1e-36c4-4093-9c7a-2478ecff68e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:05:45.612123 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a135e1e-36c4-4093-9c7a-2478ecff68e2 admin admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:05:45.612400 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a135e1e-36c4-4093-9c7a-2478ecff68e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:05:45.612672 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a135e1e-36c4-4093-9c7a-2478ecff68e2 admin admin] PATH_INFO: `/v3/domains` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:05:45.613324 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a135e1e-36c4-4093-9c7a-2478ecff68e2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 11:05:45.613746 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a135e1e-36c4-4093-9c7a-2478ecff68e2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:05:45.617218 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a135e1e-36c4-4093-9c7a-2478ecff68e2 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:05:45.628642 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 982/1980] 10.222.0.40 () {58 vars in 1153 bytes} [Fri Aug 26 11:05:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 11:05:45.633462 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13905c64-d3be-44b8-89da-5d756496b47f None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:05:45.643822 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13905c64-d3be-44b8-89da-5d756496b47f None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:05:45.655882 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75027ae9-9392-4a37-a7c1-0365c006dab9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:05:45.656511 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75027ae9-9392-4a37-a7c1-0365c006dab9 admin admin] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:05:45.656793 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75027ae9-9392-4a37-a7c1-0365c006dab9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:05:45.657055 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75027ae9-9392-4a37-a7c1-0365c006dab9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:05:45.657694 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75027ae9-9392-4a37-a7c1-0365c006dab9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:05:45.659848 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75027ae9-9392-4a37-a7c1-0365c006dab9 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:05:45.664861 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a135e1e-36c4-4093-9c7a-2478ecff68e2 admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:05:45.666763 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-7a135e1e-36c4-4093-9c7a-2478ecff68e2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:05:45.681907 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a135e1e-36c4-4093-9c7a-2478ecff68e2 admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:05:45.686587 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9a0a08-c945-4eae-909c-4e2cf87c3a21 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c277338a7f814cc6b01c5419775031bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e30ccf0c1204727b786876c141dbcd2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:05:45.687298 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9a0a08-c945-4eae-909c-4e2cf87c3a21 service cinder] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:05:45.687631 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9a0a08-c945-4eae-909c-4e2cf87c3a21 service cinder] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:05:45.687913 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9a0a08-c945-4eae-909c-4e2cf87c3a21 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:05:45.689702 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-0b9a0a08-c945-4eae-909c-4e2cf87c3a21 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:05:45.691093 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 999/1981] 10.222.0.40 () {60 vars in 1153 bytes} [Fri Aug 26 11:05:45 2022] POST /identity/v3/projects => generated 385 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 26 11:05:45.695241 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75027ae9-9392-4a37-a7c1-0365c006dab9 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:05:45.701804 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 11:05:45.715902 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 11:05:45.763723 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b9a0a08-c945-4eae-909c-4e2cf87c3a21 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:05:45.764980 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b9a0a08-c945-4eae-909c-4e2cf87c3a21 service cinder] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:05:45.773308 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75027ae9-9392-4a37-a7c1-0365c006dab9 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:05:45.774999 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 983/1982] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 11:05:45 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 26 11:05:45.778972 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b74f88f-476c-4033-9866-979db905e6c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:05:45.779720 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b74f88f-476c-4033-9866-979db905e6c9 admin admin] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:05:45.780200 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b74f88f-476c-4033-9866-979db905e6c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:05:45.780583 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b74f88f-476c-4033-9866-979db905e6c9 admin admin] PATH_INFO: `/v3/users` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:05:45.781315 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b74f88f-476c-4033-9866-979db905e6c9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:05:45.783447 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b74f88f-476c-4033-9866-979db905e6c9 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:05:45.818023 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1000/1983] 10.222.0.40 () {60 vars in 1144 bytes} [Fri Aug 26 11:05:45 2022] POST /identity/v3/users => generated 293 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 26 11:05:45.820682 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9a0a08-c945-4eae-909c-4e2cf87c3a21 service cinder] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:05:45.831946 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9a0a08-c945-4eae-909c-4e2cf87c3a21 service cinder] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:05:45.838905 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76560b84-de4a-4cfd-b0d9-26638f3be88d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:05:45.839602 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76560b84-de4a-4cfd-b0d9-26638f3be88d admin admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:05:45.839962 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76560b84-de4a-4cfd-b0d9-26638f3be88d admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:05:45.840276 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76560b84-de4a-4cfd-b0d9-26638f3be88d admin admin] PATH_INFO: `/v3/roles` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:05:45.841060 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76560b84-de4a-4cfd-b0d9-26638f3be88d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:05:45.842910 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76560b84-de4a-4cfd-b0d9-26638f3be88d admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:05:45.854960 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 984/1984] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 26 11:05:45 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 11:05:45.861401 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b74f88f-476c-4033-9866-979db905e6c9 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:05:45.873119 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b74f88f-476c-4033-9866-979db905e6c9 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:05:45.884876 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd1390f-c61b-4e7f-9294-fce84b5996ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:05:45.885655 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd1390f-c61b-4e7f-9294-fce84b5996ce admin admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:05:45.886078 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd1390f-c61b-4e7f-9294-fce84b5996ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:05:45.886456 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd1390f-c61b-4e7f-9294-fce84b5996ce admin admin] PATH_INFO: `/v3/roles` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:05:45.887369 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd1390f-c61b-4e7f-9294-fce84b5996ce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:05:45.889125 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd1390f-c61b-4e7f-9294-fce84b5996ce admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:05:45.905661 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1001/1985] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 26 11:05:45 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 11:05:45.906803 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76560b84-de4a-4cfd-b0d9-26638f3be88d admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:05:45.922164 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76560b84-de4a-4cfd-b0d9-26638f3be88d admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:05:45.933641 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9d8bab-29cc-4c6c-86a2-ea2dafbc7cd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:05:45.934341 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9d8bab-29cc-4c6c-86a2-ea2dafbc7cd2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:05:45.934616 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9d8bab-29cc-4c6c-86a2-ea2dafbc7cd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:05:45.934892 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9d8bab-29cc-4c6c-86a2-ea2dafbc7cd2 admin admin] PATH_INFO: `/v3/projects/c6322c831fbe43c4a3ed74e2df874b62/users/e984a9bd1bac42b58bf5431e3efa3e48/roles/23e174ce743b46eabe618c212691da55` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:05:45.955440 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee9d8bab-29cc-4c6c-86a2-ea2dafbc7cd2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c6322c831fbe43c4a3ed74e2df874b62, user_id=e984a9bd1bac42b58bf5431e3efa3e48, role_id=23e174ce743b46eabe618c212691da55)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:05:45.958321 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee9d8bab-29cc-4c6c-86a2-ea2dafbc7cd2 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:05:45.977097 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 985/1986] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 26 11:05:45 2022] PUT /identity/v3/projects/c6322c831fbe43c4a3ed74e2df874b62/users/e984a9bd1bac42b58bf5431e3efa3e48/roles/23e174ce743b46eabe618c212691da55 => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 11:05:45.989845 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987d12e3-1fb9-48df-9d6d-93d8d6b44111 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:05:45.990289 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987d12e3-1fb9-48df-9d6d-93d8d6b44111 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:05:45.990562 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987d12e3-1fb9-48df-9d6d-93d8d6b44111 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:05:46.035430 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-987d12e3-1fb9-48df-9d6d-93d8d6b44111 None None] MFA Rules not processed for user `e984a9bd1bac42b58bf5431e3efa3e48`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 11:05:46.037911 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-987d12e3-1fb9-48df-9d6d-93d8d6b44111 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:05:46.094721 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9d8bab-29cc-4c6c-86a2-ea2dafbc7cd2 admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:05:46.096682 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-ee9d8bab-29cc-4c6c-86a2-ea2dafbc7cd2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:05:46.137522 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1002/1987] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 11:05:45 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 11:05:46.142457 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987d12e3-1fb9-48df-9d6d-93d8d6b44111 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:05:46.183359 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9d8bab-29cc-4c6c-86a2-ea2dafbc7cd2 admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:05:46.188374 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e0ea55-9360-49b0-99da-7b43443345e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:05:46.189015 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e0ea55-9360-49b0-99da-7b43443345e5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:05:46.189456 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e0ea55-9360-49b0-99da-7b43443345e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:05:46.189734 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e0ea55-9360-49b0-99da-7b43443345e5 admin admin] PATH_INFO: `/v3/users/f5e361c7ef1745ae9dc2b499a591206b` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:05:46.190937 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e0ea55-9360-49b0-99da-7b43443345e5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f5e361c7ef1745ae9dc2b499a591206b)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:05:46.192904 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e0ea55-9360-49b0-99da-7b43443345e5 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:05:46.212364 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987d12e3-1fb9-48df-9d6d-93d8d6b44111 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:05:46.219681 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e70b2f-ae9c-4b47-acc3-2bc0580c3129 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:05:46.220435 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e70b2f-ae9c-4b47-acc3-2bc0580c3129 admin admin] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:05:46.220961 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e70b2f-ae9c-4b47-acc3-2bc0580c3129 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:05:46.221487 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e70b2f-ae9c-4b47-acc3-2bc0580c3129 admin admin] PATH_INFO: `/v3/projects` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:05:46.222251 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e70b2f-ae9c-4b47-acc3-2bc0580c3129 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:05:46.224244 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e70b2f-ae9c-4b47-acc3-2bc0580c3129 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:05:46.260659 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1003/1988] 10.222.0.40 () {60 vars in 1153 bytes} [Fri Aug 26 11:05:46 2022] POST /identity/v3/projects => generated 387 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 11:05:46.264167 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e70b2f-ae9c-4b47-acc3-2bc0580c3129 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:05:46.272769 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-c5e0ea55-9360-49b0-99da-7b43443345e5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f5e361c7ef1745ae9dc2b499a591206b'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:05:46.280689 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 11:05:46.285255 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-c5e0ea55-9360-49b0-99da-7b43443345e5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f5e361c7ef1745ae9dc2b499a591206b'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:05:46.286319 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-c5e0ea55-9360-49b0-99da-7b43443345e5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f5e361c7ef1745ae9dc2b499a591206b'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:05:46.314219 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-c5e0ea55-9360-49b0-99da-7b43443345e5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f5e361c7ef1745ae9dc2b499a591206b'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:05:46.315084 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-c5e0ea55-9360-49b0-99da-7b43443345e5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f5e361c7ef1745ae9dc2b499a591206b'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:05:46.322886 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 11:05:46.345457 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 986/1989] 10.222.0.40 () {58 vars in 1224 bytes} [Fri Aug 26 11:05:46 2022] DELETE /identity/v3/users/f5e361c7ef1745ae9dc2b499a591206b => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 26 11:05:46.348833 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1813bed2-64da-4e57-b276-2530e1c6e592 None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:05:46.349117 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1813bed2-64da-4e57-b276-2530e1c6e592 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:05:46.349956 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1813bed2-64da-4e57-b276-2530e1c6e592 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:05:46.384580 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-1813bed2-64da-4e57-b276-2530e1c6e592 None None] MFA Rules not processed for user `ca155a48e2024c32af3fd2e19e5fac11`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 11:05:46.389759 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-1813bed2-64da-4e57-b276-2530e1c6e592 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:05:46.399375 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e70b2f-ae9c-4b47-acc3-2bc0580c3129 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:05:46.496571 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 987/1990] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 11:05:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 11:05:46.505102 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1813bed2-64da-4e57-b276-2530e1c6e592 None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:05:46.506800 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1570564c-36fe-4285-8352-bf3d8277ff89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:05:46.506939 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-1813bed2-64da-4e57-b276-2530e1c6e592 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:05:46.507490 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1570564c-36fe-4285-8352-bf3d8277ff89 admin admin] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:05:46.507771 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1570564c-36fe-4285-8352-bf3d8277ff89 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:05:46.508046 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1570564c-36fe-4285-8352-bf3d8277ff89 admin admin] PATH_INFO: `/v3/users` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:05:46.508687 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1570564c-36fe-4285-8352-bf3d8277ff89 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:05:46.510650 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1570564c-36fe-4285-8352-bf3d8277ff89 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:05:46.535804 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1004/1991] 10.222.0.40 () {60 vars in 1144 bytes} [Fri Aug 26 11:05:46 2022] POST /identity/v3/users => generated 300 bytes in 272 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 11:05:46.540941 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1570564c-36fe-4285-8352-bf3d8277ff89 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:05:46.542799 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-1570564c-36fe-4285-8352-bf3d8277ff89 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:05:46.554939 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1570564c-36fe-4285-8352-bf3d8277ff89 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:05:46.560163 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42d3253-6199-4bd5-a588-32de1243dfa0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:05:46.560783 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42d3253-6199-4bd5-a588-32de1243dfa0 admin admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:05:46.561085 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42d3253-6199-4bd5-a588-32de1243dfa0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:05:46.561314 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42d3253-6199-4bd5-a588-32de1243dfa0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:05:46.561962 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d42d3253-6199-4bd5-a588-32de1243dfa0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:05:46.563590 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d42d3253-6199-4bd5-a588-32de1243dfa0 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:05:46.572277 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1813bed2-64da-4e57-b276-2530e1c6e592 None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:05:46.578233 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1005/1992] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 26 11:05:46 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 11:05:46.579460 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42d3253-6199-4bd5-a588-32de1243dfa0 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:05:46.579605 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1971c24-a101-4227-911f-30f34163ea34 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91da2acd2534423091537e1ba068d2b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e30ccf0c1204727b786876c141dbcd2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:05:46.580097 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1971c24-a101-4227-911f-30f34163ea34 service neutron] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:05:46.580376 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1971c24-a101-4227-911f-30f34163ea34 service neutron] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:05:46.580646 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1971c24-a101-4227-911f-30f34163ea34 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:05:46.593096 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1971c24-a101-4227-911f-30f34163ea34 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:05:46.594183 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42d3253-6199-4bd5-a588-32de1243dfa0 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:05:46.594596 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1971c24-a101-4227-911f-30f34163ea34 service neutron] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:05:46.599195 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f91fa7-8e59-499d-a2e8-4b88d7c902dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:05:46.601030 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f91fa7-8e59-499d-a2e8-4b88d7c902dd admin admin] REQUEST_METHOD: `PUT` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:05:46.601309 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f91fa7-8e59-499d-a2e8-4b88d7c902dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:05:46.601722 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f91fa7-8e59-499d-a2e8-4b88d7c902dd admin admin] PATH_INFO: `/v3/projects/25a0a1f62ffb40e5b512e523924850e6/users/0e2b74b185354ea0acf83c6de207ec7f/roles/1f97e4602e2e4d6498913a81f9ae8d6a` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:05:46.605071 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 988/1993] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 11:05:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 26 11:05:46.620948 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f91fa7-8e59-499d-a2e8-4b88d7c902dd admin admin] RBAC: Authorizing `identity:create_grant(project_id=25a0a1f62ffb40e5b512e523924850e6, user_id=0e2b74b185354ea0acf83c6de207ec7f, role_id=1f97e4602e2e4d6498913a81f9ae8d6a)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:05:46.623630 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f91fa7-8e59-499d-a2e8-4b88d7c902dd admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:05:46.653376 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1006/1994] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 26 11:05:46 2022] PUT /identity/v3/projects/25a0a1f62ffb40e5b512e523924850e6/users/0e2b74b185354ea0acf83c6de207ec7f/roles/1f97e4602e2e4d6498913a81f9ae8d6a => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 26 11:05:46.654364 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1971c24-a101-4227-911f-30f34163ea34 service neutron] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:05:46.668826 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1971c24-a101-4227-911f-30f34163ea34 service neutron] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:05:46.754769 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046b5702-1c52-46dc-b7e7-fb55cfe2d9d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:05:46.756588 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046b5702-1c52-46dc-b7e7-fb55cfe2d9d4 admin admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:05:46.756981 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046b5702-1c52-46dc-b7e7-fb55cfe2d9d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:05:46.757473 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046b5702-1c52-46dc-b7e7-fb55cfe2d9d4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:05:46.758711 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-046b5702-1c52-46dc-b7e7-fb55cfe2d9d4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:05:46.761184 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-046b5702-1c52-46dc-b7e7-fb55cfe2d9d4 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:05:46.779687 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f91fa7-8e59-499d-a2e8-4b88d7c902dd admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:05:46.787969 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 989/1995] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 26 11:05:46 2022] GET /identity/v3/roles => generated 1420 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 26 11:05:46.796709 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f91fa7-8e59-499d-a2e8-4b88d7c902dd admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:05:46.805214 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a66f1b6-21ef-438c-9ff6-e027a15de7b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:05:46.805900 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a66f1b6-21ef-438c-9ff6-e027a15de7b2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:05:46.806233 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a66f1b6-21ef-438c-9ff6-e027a15de7b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:05:46.806563 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a66f1b6-21ef-438c-9ff6-e027a15de7b2 admin admin] PATH_INFO: `/v3/projects/25a0a1f62ffb40e5b512e523924850e6/users/0e2b74b185354ea0acf83c6de207ec7f/roles/1f97e4602e2e4d6498913a81f9ae8d6a` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:05:46.810496 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a66f1b6-21ef-438c-9ff6-e027a15de7b2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=25a0a1f62ffb40e5b512e523924850e6, user_id=0e2b74b185354ea0acf83c6de207ec7f, role_id=1f97e4602e2e4d6498913a81f9ae8d6a)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:05:46.825934 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a66f1b6-21ef-438c-9ff6-e027a15de7b2 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:05:46.851276 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fb9e66-b4f7-4171-b7c4-71820855f9b5 None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:05:46.851564 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fb9e66-b4f7-4171-b7c4-71820855f9b5 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:05:46.851863 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fb9e66-b4f7-4171-b7c4-71820855f9b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:05:46.864862 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1007/1996] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 26 11:05:46 2022] PUT /identity/v3/projects/25a0a1f62ffb40e5b512e523924850e6/users/0e2b74b185354ea0acf83c6de207ec7f/roles/1f97e4602e2e4d6498913a81f9ae8d6a => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 11:05:46.899136 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-74fb9e66-b4f7-4171-b7c4-71820855f9b5 None None] MFA Rules not processed for user `0e2b74b185354ea0acf83c6de207ec7f`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 11:05:46.900920 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-74fb9e66-b4f7-4171-b7c4-71820855f9b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:05:46.936914 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 11:05:46.970580 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a66f1b6-21ef-438c-9ff6-e027a15de7b2 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:05:46.972972 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-6a66f1b6-21ef-438c-9ff6-e027a15de7b2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:05:47.014649 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 990/1997] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 11:05:46 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 11:05:47.024794 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74fb9e66-b4f7-4171-b7c4-71820855f9b5 None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:05:47.027306 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-74fb9e66-b4f7-4171-b7c4-71820855f9b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:05:47.055154 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 11:05:47.060134 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a66f1b6-21ef-438c-9ff6-e027a15de7b2 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:05:47.067414 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef606b1-c634-4058-bfcd-4a2dda2b68a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:05:47.068613 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef606b1-c634-4058-bfcd-4a2dda2b68a9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:05:47.069155 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef606b1-c634-4058-bfcd-4a2dda2b68a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:05:47.069322 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef606b1-c634-4058-bfcd-4a2dda2b68a9 admin admin] PATH_INFO: `/v3/projects/950827994dd14b86b76a399de03b563b` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:05:47.070576 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef606b1-c634-4058-bfcd-4a2dda2b68a9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=950827994dd14b86b76a399de03b563b)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:05:47.072873 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef606b1-c634-4058-bfcd-4a2dda2b68a9 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:05:47.098702 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74fb9e66-b4f7-4171-b7c4-71820855f9b5 None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:05:47.103974 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c455df-2bca-40e7-b5d7-04ae6152585c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c277338a7f814cc6b01c5419775031bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e30ccf0c1204727b786876c141dbcd2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:05:47.104612 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c455df-2bca-40e7-b5d7-04ae6152585c service cinder] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:05:47.104865 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c455df-2bca-40e7-b5d7-04ae6152585c service cinder] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:05:47.105132 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c455df-2bca-40e7-b5d7-04ae6152585c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:05:47.116111 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c455df-2bca-40e7-b5d7-04ae6152585c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:05:47.117923 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c455df-2bca-40e7-b5d7-04ae6152585c service cinder] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:05:47.128096 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 991/1998] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 11:05:47 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 26 11:05:47.129360 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-cef606b1-c634-4058-bfcd-4a2dda2b68a9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '950827994dd14b86b76a399de03b563b', 'request_id': 'req-cef606b1-c634-4058-bfcd-4a2dda2b68a9'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:05:47.137519 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-cef606b1-c634-4058-bfcd-4a2dda2b68a9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '950827994dd14b86b76a399de03b563b', 'request_id': 'req-cef606b1-c634-4058-bfcd-4a2dda2b68a9'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:05:47.143975 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-cef606b1-c634-4058-bfcd-4a2dda2b68a9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '950827994dd14b86b76a399de03b563b', 'request_id': 'req-cef606b1-c634-4058-bfcd-4a2dda2b68a9'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:05:47.158126 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-cef606b1-c634-4058-bfcd-4a2dda2b68a9 admin admin] The token cache is being invalidate because project 950827994dd14b86b76a399de03b563b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81424) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 11:05:47.158361 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-cef606b1-c634-4058-bfcd-4a2dda2b68a9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:05:47.159016 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-cef606b1-c634-4058-bfcd-4a2dda2b68a9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:05:47.160769 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1008/1999] 10.222.0.40 () {58 vars in 1233 bytes} [Fri Aug 26 11:05:46 2022] DELETE /identity/v3/projects/950827994dd14b86b76a399de03b563b => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 26 11:05:47.167143 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c455df-2bca-40e7-b5d7-04ae6152585c service cinder] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:05:47.180026 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c455df-2bca-40e7-b5d7-04ae6152585c service cinder] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:05:47.251538 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe0742f-c699-4ca7-818b-a9c732a5b396 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:05:47.252214 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe0742f-c699-4ca7-818b-a9c732a5b396 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:05:47.252497 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe0742f-c699-4ca7-818b-a9c732a5b396 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:05:47.252993 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe0742f-c699-4ca7-818b-a9c732a5b396 admin admin] PATH_INFO: `/v3/users/7411c7058ca74601a2806675ab2ae179` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:05:47.254299 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe0742f-c699-4ca7-818b-a9c732a5b396 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7411c7058ca74601a2806675ab2ae179)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:05:47.257280 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe0742f-c699-4ca7-818b-a9c732a5b396 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:05:47.311030 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-fbe0742f-c699-4ca7-818b-a9c732a5b396 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7411c7058ca74601a2806675ab2ae179'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:05:47.323362 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-fbe0742f-c699-4ca7-818b-a9c732a5b396 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7411c7058ca74601a2806675ab2ae179'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:05:47.324140 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-fbe0742f-c699-4ca7-818b-a9c732a5b396 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7411c7058ca74601a2806675ab2ae179'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:05:47.332591 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=92,keystone:DELETE=8 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 11:05:47.340924 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-fbe0742f-c699-4ca7-818b-a9c732a5b396 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7411c7058ca74601a2806675ab2ae179'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:05:47.341957 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-fbe0742f-c699-4ca7-818b-a9c732a5b396 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7411c7058ca74601a2806675ab2ae179'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:05:47.371306 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 992/2000] 10.222.0.40 () {58 vars in 1224 bytes} [Fri Aug 26 11:05:47 2022] DELETE /identity/v3/users/7411c7058ca74601a2806675ab2ae179 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 11:05:47.700381 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef606b1-c634-4058-bfcd-4a2dda2b68a9 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:05:47.701949 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-cef606b1-c634-4058-bfcd-4a2dda2b68a9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:05:47.723883 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef606b1-c634-4058-bfcd-4a2dda2b68a9 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:05:47.797849 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e93c603-738a-4eaa-a0d1-630c82edec36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:05:47.799089 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e93c603-738a-4eaa-a0d1-630c82edec36 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:05:47.799521 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e93c603-738a-4eaa-a0d1-630c82edec36 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:05:47.799939 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e93c603-738a-4eaa-a0d1-630c82edec36 admin admin] PATH_INFO: `/v3/projects/0e6d7f8fe42d4678a3cfffdb631cac4d` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:05:47.803061 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e93c603-738a-4eaa-a0d1-630c82edec36 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0e6d7f8fe42d4678a3cfffdb631cac4d)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:05:47.805279 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e93c603-738a-4eaa-a0d1-630c82edec36 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:05:47.885133 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-1e93c603-738a-4eaa-a0d1-630c82edec36 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0e6d7f8fe42d4678a3cfffdb631cac4d', 'request_id': 'req-1e93c603-738a-4eaa-a0d1-630c82edec36'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:05:47.892374 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-1e93c603-738a-4eaa-a0d1-630c82edec36 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0e6d7f8fe42d4678a3cfffdb631cac4d', 'request_id': 'req-1e93c603-738a-4eaa-a0d1-630c82edec36'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:05:47.902680 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-1e93c603-738a-4eaa-a0d1-630c82edec36 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0e6d7f8fe42d4678a3cfffdb631cac4d', 'request_id': 'req-1e93c603-738a-4eaa-a0d1-630c82edec36'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:05:47.919495 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-1e93c603-738a-4eaa-a0d1-630c82edec36 admin admin] The token cache is being invalidate because project 0e6d7f8fe42d4678a3cfffdb631cac4d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81424) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 11:05:47.919795 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-1e93c603-738a-4eaa-a0d1-630c82edec36 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:05:47.920475 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-1e93c603-738a-4eaa-a0d1-630c82edec36 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:05:47.922454 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1009/2001] 10.222.0.40 () {58 vars in 1233 bytes} [Fri Aug 26 11:05:47 2022] DELETE /identity/v3/projects/0e6d7f8fe42d4678a3cfffdb631cac4d => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 11:05:47.927638 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274e3b59-ad0a-41ed-895e-888b2b131b89 None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:05:47.928133 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274e3b59-ad0a-41ed-895e-888b2b131b89 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:05:47.928582 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274e3b59-ad0a-41ed-895e-888b2b131b89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:05:47.942632 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e93c603-738a-4eaa-a0d1-630c82edec36 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:05:47.944302 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-1e93c603-738a-4eaa-a0d1-630c82edec36 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:05:47.967904 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-274e3b59-ad0a-41ed-895e-888b2b131b89 None None] MFA Rules not processed for user `ca155a48e2024c32af3fd2e19e5fac11`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 11:05:47.970142 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-274e3b59-ad0a-41ed-895e-888b2b131b89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:05:47.975057 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 11:05:48.050339 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 993/2002] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 11:05:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 11:05:48.061007 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596cff57-f8b5-4b85-8790-af77393e3e3c None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:05:48.061302 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596cff57-f8b5-4b85-8790-af77393e3e3c None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:05:48.061579 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596cff57-f8b5-4b85-8790-af77393e3e3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:05:48.075551 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e93c603-738a-4eaa-a0d1-630c82edec36 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:05:48.081212 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073193e0-51a2-49c7-9d8a-563900090785 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:05:48.082278 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073193e0-51a2-49c7-9d8a-563900090785 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:05:48.082922 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-596cff57-f8b5-4b85-8790-af77393e3e3c None None] MFA Rules not processed for user `ca155a48e2024c32af3fd2e19e5fac11`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 11:05:48.083182 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073193e0-51a2-49c7-9d8a-563900090785 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:05:48.083661 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073193e0-51a2-49c7-9d8a-563900090785 admin admin] PATH_INFO: `/v3/users/e984a9bd1bac42b58bf5431e3efa3e48` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:05:48.084771 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-596cff57-f8b5-4b85-8790-af77393e3e3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:05:48.085790 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-073193e0-51a2-49c7-9d8a-563900090785 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e984a9bd1bac42b58bf5431e3efa3e48)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:05:48.088119 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-073193e0-51a2-49c7-9d8a-563900090785 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:05:48.094282 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 994/2003] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 11:05:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 11:05:48.098495 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596cff57-f8b5-4b85-8790-af77393e3e3c None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:05:48.109281 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596cff57-f8b5-4b85-8790-af77393e3e3c None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:05:48.114073 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61fe2b11-7099-40e6-95ee-804a096c16aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:05:48.114686 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fe2b11-7099-40e6-95ee-804a096c16aa admin admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:05:48.115024 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fe2b11-7099-40e6-95ee-804a096c16aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:05:48.115347 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fe2b11-7099-40e6-95ee-804a096c16aa admin admin] PATH_INFO: `/v3/domains` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:05:48.116265 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61fe2b11-7099-40e6-95ee-804a096c16aa admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 11:05:48.116629 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61fe2b11-7099-40e6-95ee-804a096c16aa admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:05:48.118630 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61fe2b11-7099-40e6-95ee-804a096c16aa admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:05:48.135535 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 995/2004] 10.222.0.40 () {58 vars in 1153 bytes} [Fri Aug 26 11:05:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 11:05:48.136950 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61fe2b11-7099-40e6-95ee-804a096c16aa admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:05:48.141327 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-073193e0-51a2-49c7-9d8a-563900090785 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e984a9bd1bac42b58bf5431e3efa3e48'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:05:48.142126 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-073193e0-51a2-49c7-9d8a-563900090785 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e984a9bd1bac42b58bf5431e3efa3e48'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:05:48.148811 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61fe2b11-7099-40e6-95ee-804a096c16aa admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:05:48.150295 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-073193e0-51a2-49c7-9d8a-563900090785 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e984a9bd1bac42b58bf5431e3efa3e48'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:05:48.150889 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-073193e0-51a2-49c7-9d8a-563900090785 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e984a9bd1bac42b58bf5431e3efa3e48'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:05:48.153555 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36bf02b-4a65-45dc-89ea-5aecbe2a344f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:05:48.154259 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36bf02b-4a65-45dc-89ea-5aecbe2a344f admin admin] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:05:48.154536 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36bf02b-4a65-45dc-89ea-5aecbe2a344f admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:05:48.154803 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36bf02b-4a65-45dc-89ea-5aecbe2a344f admin admin] PATH_INFO: `/v3/projects` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:05:48.155419 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d36bf02b-4a65-45dc-89ea-5aecbe2a344f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:05:48.157302 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d36bf02b-4a65-45dc-89ea-5aecbe2a344f admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:05:48.164405 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-073193e0-51a2-49c7-9d8a-563900090785 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e984a9bd1bac42b58bf5431e3efa3e48'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:05:48.182109 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1010/2005] 10.222.0.40 () {58 vars in 1224 bytes} [Fri Aug 26 11:05:47 2022] DELETE /identity/v3/users/e984a9bd1bac42b58bf5431e3efa3e48 => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 11:05:48.186723 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3bbd131-efbf-4d2b-917d-23475b262813 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:05:48.186835 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3bbd131-efbf-4d2b-917d-23475b262813 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:05:48.187267 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3bbd131-efbf-4d2b-917d-23475b262813 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:05:48.195540 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 996/2006] 10.222.0.40 () {60 vars in 1153 bytes} [Fri Aug 26 11:05:48 2022] POST /identity/v3/projects => generated 371 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 11:05:48.196817 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36bf02b-4a65-45dc-89ea-5aecbe2a344f admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:05:48.198500 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-d36bf02b-4a65-45dc-89ea-5aecbe2a344f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:05:48.207527 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-b3bbd131-efbf-4d2b-917d-23475b262813 None None] MFA Rules not processed for user `ca155a48e2024c32af3fd2e19e5fac11`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 11:05:48.209290 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-b3bbd131-efbf-4d2b-917d-23475b262813 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:05:48.274900 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1011/2007] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 11:05:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 11:05:48.288744 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3bbd131-efbf-4d2b-917d-23475b262813 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:05:48.292616 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-b3bbd131-efbf-4d2b-917d-23475b262813 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:05:48.301590 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36bf02b-4a65-45dc-89ea-5aecbe2a344f admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:05:48.306928 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db502f38-7245-4d73-808b-8d21566d804c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:05:48.307559 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db502f38-7245-4d73-808b-8d21566d804c admin admin] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:05:48.307840 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db502f38-7245-4d73-808b-8d21566d804c admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:05:48.308116 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db502f38-7245-4d73-808b-8d21566d804c admin admin] PATH_INFO: `/v3/users` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:05:48.308878 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db502f38-7245-4d73-808b-8d21566d804c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:05:48.310898 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db502f38-7245-4d73-808b-8d21566d804c admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:05:48.339262 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 997/2008] 10.222.0.40 () {60 vars in 1144 bytes} [Fri Aug 26 11:05:48 2022] POST /identity/v3/users => generated 286 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 11:05:48.340514 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db502f38-7245-4d73-808b-8d21566d804c admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:05:48.350925 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db502f38-7245-4d73-808b-8d21566d804c admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:05:48.352037 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 11:05:48.356110 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95052cf4-a835-4483-bcea-49ec5df2a0a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:05:48.356786 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95052cf4-a835-4483-bcea-49ec5df2a0a7 admin admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:05:48.357072 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95052cf4-a835-4483-bcea-49ec5df2a0a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:05:48.357349 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95052cf4-a835-4483-bcea-49ec5df2a0a7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:05:48.358096 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95052cf4-a835-4483-bcea-49ec5df2a0a7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:05:48.359714 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95052cf4-a835-4483-bcea-49ec5df2a0a7 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:05:48.372186 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 998/2009] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 26 11:05:48 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 11:05:48.372668 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3bbd131-efbf-4d2b-917d-23475b262813 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:05:48.376056 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95052cf4-a835-4483-bcea-49ec5df2a0a7 admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:05:48.377083 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1b315b-e7c8-4b7b-b2c7-e595240c42d8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91da2acd2534423091537e1ba068d2b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e30ccf0c1204727b786876c141dbcd2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:05:48.377746 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1b315b-e7c8-4b7b-b2c7-e595240c42d8 service neutron] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:05:48.378053 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1b315b-e7c8-4b7b-b2c7-e595240c42d8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:05:48.378580 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1b315b-e7c8-4b7b-b2c7-e595240c42d8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:05:48.389035 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95052cf4-a835-4483-bcea-49ec5df2a0a7 admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:05:48.391177 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a1b315b-e7c8-4b7b-b2c7-e595240c42d8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:05:48.392590 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a1b315b-e7c8-4b7b-b2c7-e595240c42d8 service neutron] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:05:48.394509 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2eb662-4ee8-45e0-8a7c-6273a6a61384 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:05:48.395119 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2eb662-4ee8-45e0-8a7c-6273a6a61384 admin admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:05:48.395529 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2eb662-4ee8-45e0-8a7c-6273a6a61384 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:05:48.395813 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2eb662-4ee8-45e0-8a7c-6273a6a61384 admin admin] PATH_INFO: `/v3/roles` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:05:48.396532 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb2eb662-4ee8-45e0-8a7c-6273a6a61384 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:05:48.398222 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb2eb662-4ee8-45e0-8a7c-6273a6a61384 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:05:48.404937 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1012/2010] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 11:05:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 11:05:48.419085 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 999/2011] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 26 11:05:48 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 26 11:05:48.426734 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1b315b-e7c8-4b7b-b2c7-e595240c42d8 service neutron] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:05:48.440067 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1b315b-e7c8-4b7b-b2c7-e595240c42d8 service neutron] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:05:48.445448 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94df5a27-9f8d-491d-86f9-5c166a2b109b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:05:48.447165 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94df5a27-9f8d-491d-86f9-5c166a2b109b admin admin] REQUEST_METHOD: `PUT` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:05:48.447165 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94df5a27-9f8d-491d-86f9-5c166a2b109b admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:05:48.447613 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94df5a27-9f8d-491d-86f9-5c166a2b109b admin admin] PATH_INFO: `/v3/projects/48d502e519f4425a8602bc040762fe47/users/d9b0982290b346ac9967bf57ab5cdba1/roles/23e174ce743b46eabe618c212691da55` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:05:48.475783 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94df5a27-9f8d-491d-86f9-5c166a2b109b admin admin] RBAC: Authorizing `identity:create_grant(project_id=48d502e519f4425a8602bc040762fe47, user_id=d9b0982290b346ac9967bf57ab5cdba1, role_id=23e174ce743b46eabe618c212691da55)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:05:48.481274 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94df5a27-9f8d-491d-86f9-5c166a2b109b admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:05:48.514660 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1013/2012] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 26 11:05:48 2022] PUT /identity/v3/projects/48d502e519f4425a8602bc040762fe47/users/d9b0982290b346ac9967bf57ab5cdba1/roles/23e174ce743b46eabe618c212691da55 => generated 0 bytes in 88 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 11:05:48.520033 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a1fbc3-082f-44b1-9956-5e483ff893d0 None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:05:48.520291 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a1fbc3-082f-44b1-9956-5e483ff893d0 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:05:48.521348 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a1fbc3-082f-44b1-9956-5e483ff893d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:05:48.545053 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-84a1fbc3-082f-44b1-9956-5e483ff893d0 None None] MFA Rules not processed for user `ca155a48e2024c32af3fd2e19e5fac11`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 11:05:48.546451 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-84a1fbc3-082f-44b1-9956-5e483ff893d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:05:48.550979 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=93,keystone:DELETE=2,keystone:INSERT=5 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 11:05:48.664417 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1000/2013] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 11:05:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 11:05:48.678934 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94df5a27-9f8d-491d-86f9-5c166a2b109b admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:05:48.681623 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-94df5a27-9f8d-491d-86f9-5c166a2b109b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:05:48.759116 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94df5a27-9f8d-491d-86f9-5c166a2b109b admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:05:48.763227 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50afd030-2c1a-4b62-bdac-f1bd0960a7fe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91da2acd2534423091537e1ba068d2b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e30ccf0c1204727b786876c141dbcd2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:05:48.763978 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50afd030-2c1a-4b62-bdac-f1bd0960a7fe service neutron] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:05:48.764520 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50afd030-2c1a-4b62-bdac-f1bd0960a7fe service neutron] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:05:48.764869 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50afd030-2c1a-4b62-bdac-f1bd0960a7fe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:05:48.778289 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50afd030-2c1a-4b62-bdac-f1bd0960a7fe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:05:48.779852 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50afd030-2c1a-4b62-bdac-f1bd0960a7fe service neutron] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:05:48.804916 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1014/2014] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 11:05:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 11:05:48.845636 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a1fbc3-082f-44b1-9956-5e483ff893d0 None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:05:48.847697 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-84a1fbc3-082f-44b1-9956-5e483ff893d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:05:48.878293 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a1fbc3-082f-44b1-9956-5e483ff893d0 None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:05:48.900116 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0795a4-7d6b-4405-b799-04e4449d54d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:05:48.901236 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0795a4-7d6b-4405-b799-04e4449d54d7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:05:48.901236 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0795a4-7d6b-4405-b799-04e4449d54d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:05:48.901442 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0795a4-7d6b-4405-b799-04e4449d54d7 admin admin] PATH_INFO: `/v3/projects/c6322c831fbe43c4a3ed74e2df874b62` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:05:48.902645 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca0795a4-7d6b-4405-b799-04e4449d54d7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c6322c831fbe43c4a3ed74e2df874b62)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:05:48.904665 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca0795a4-7d6b-4405-b799-04e4449d54d7 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:05:48.988844 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-ca0795a4-7d6b-4405-b799-04e4449d54d7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c6322c831fbe43c4a3ed74e2df874b62', 'request_id': 'req-ca0795a4-7d6b-4405-b799-04e4449d54d7'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:05:49.007813 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-ca0795a4-7d6b-4405-b799-04e4449d54d7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c6322c831fbe43c4a3ed74e2df874b62', 'request_id': 'req-ca0795a4-7d6b-4405-b799-04e4449d54d7'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:05:49.019142 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-ca0795a4-7d6b-4405-b799-04e4449d54d7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c6322c831fbe43c4a3ed74e2df874b62', 'request_id': 'req-ca0795a4-7d6b-4405-b799-04e4449d54d7'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:05:49.041158 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-ca0795a4-7d6b-4405-b799-04e4449d54d7 admin admin] The token cache is being invalidate because project c6322c831fbe43c4a3ed74e2df874b62 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81423) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 11:05:49.041378 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-ca0795a4-7d6b-4405-b799-04e4449d54d7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:05:49.042510 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-ca0795a4-7d6b-4405-b799-04e4449d54d7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:05:49.044696 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1001/2015] 10.222.0.40 () {58 vars in 1233 bytes} [Fri Aug 26 11:05:48 2022] DELETE /identity/v3/projects/c6322c831fbe43c4a3ed74e2df874b62 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 11:05:49.048590 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50afd030-2c1a-4b62-bdac-f1bd0960a7fe service neutron] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:05:49.061880 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50afd030-2c1a-4b62-bdac-f1bd0960a7fe service neutron] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:05:49.142441 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d784e145-36c1-48a8-a962-204995e74e06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:05:49.143296 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d784e145-36c1-48a8-a962-204995e74e06 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:05:49.143588 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d784e145-36c1-48a8-a962-204995e74e06 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:05:49.143991 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d784e145-36c1-48a8-a962-204995e74e06 admin admin] PATH_INFO: `/v3/users/0e2b74b185354ea0acf83c6de207ec7f` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:05:49.145192 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d784e145-36c1-48a8-a962-204995e74e06 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0e2b74b185354ea0acf83c6de207ec7f)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:05:49.148026 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d784e145-36c1-48a8-a962-204995e74e06 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:05:49.197766 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-d784e145-36c1-48a8-a962-204995e74e06 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0e2b74b185354ea0acf83c6de207ec7f'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:05:49.199899 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-d784e145-36c1-48a8-a962-204995e74e06 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0e2b74b185354ea0acf83c6de207ec7f'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:05:49.208494 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-d784e145-36c1-48a8-a962-204995e74e06 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0e2b74b185354ea0acf83c6de207ec7f'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:05:49.210169 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-d784e145-36c1-48a8-a962-204995e74e06 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0e2b74b185354ea0acf83c6de207ec7f'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:05:49.215661 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=92,keystone:INSERT=1,keystone:DELETE=7 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 11:05:49.227160 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-d784e145-36c1-48a8-a962-204995e74e06 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0e2b74b185354ea0acf83c6de207ec7f'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:05:49.240592 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1015/2016] 10.222.0.40 () {58 vars in 1224 bytes} [Fri Aug 26 11:05:49 2022] DELETE /identity/v3/users/0e2b74b185354ea0acf83c6de207ec7f => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 11:05:49.543958 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0795a4-7d6b-4405-b799-04e4449d54d7 admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:05:49.545613 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-ca0795a4-7d6b-4405-b799-04e4449d54d7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:05:49.561039 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0795a4-7d6b-4405-b799-04e4449d54d7 admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:05:49.635605 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c73ea0-54f0-4e0c-b4ee-70ddf463c405 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:05:49.636266 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c73ea0-54f0-4e0c-b4ee-70ddf463c405 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:05:49.637071 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c73ea0-54f0-4e0c-b4ee-70ddf463c405 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:05:49.637071 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c73ea0-54f0-4e0c-b4ee-70ddf463c405 admin admin] PATH_INFO: `/v3/projects/25a0a1f62ffb40e5b512e523924850e6` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:05:49.637941 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c73ea0-54f0-4e0c-b4ee-70ddf463c405 admin admin] RBAC: Authorizing `identity:delete_project(project_id=25a0a1f62ffb40e5b512e523924850e6)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:05:49.639892 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c73ea0-54f0-4e0c-b4ee-70ddf463c405 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:05:49.660377 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90b568e-1362-42e3-8d05-9ed510ad88cc None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:05:49.660670 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90b568e-1362-42e3-8d05-9ed510ad88cc None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:05:49.660977 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90b568e-1362-42e3-8d05-9ed510ad88cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:05:49.700156 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-b90b568e-1362-42e3-8d05-9ed510ad88cc None None] MFA Rules not processed for user `d9b0982290b346ac9967bf57ab5cdba1`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 11:05:49.701753 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-b90b568e-1362-42e3-8d05-9ed510ad88cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:05:49.724134 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 11:05:49.738261 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-d4c73ea0-54f0-4e0c-b4ee-70ddf463c405 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '25a0a1f62ffb40e5b512e523924850e6', 'request_id': 'req-d4c73ea0-54f0-4e0c-b4ee-70ddf463c405'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:05:49.749234 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-d4c73ea0-54f0-4e0c-b4ee-70ddf463c405 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '25a0a1f62ffb40e5b512e523924850e6', 'request_id': 'req-d4c73ea0-54f0-4e0c-b4ee-70ddf463c405'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:05:49.758175 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-d4c73ea0-54f0-4e0c-b4ee-70ddf463c405 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '25a0a1f62ffb40e5b512e523924850e6', 'request_id': 'req-d4c73ea0-54f0-4e0c-b4ee-70ddf463c405'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:05:49.781758 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-d4c73ea0-54f0-4e0c-b4ee-70ddf463c405 admin admin] The token cache is being invalidate because project 25a0a1f62ffb40e5b512e523924850e6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81423) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 11:05:49.782161 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-d4c73ea0-54f0-4e0c-b4ee-70ddf463c405 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:05:49.782982 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-d4c73ea0-54f0-4e0c-b4ee-70ddf463c405 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:05:49.786084 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1002/2017] 10.222.0.40 () {58 vars in 1233 bytes} [Fri Aug 26 11:05:49 2022] DELETE /identity/v3/projects/25a0a1f62ffb40e5b512e523924850e6 => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 11:05:49.790410 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d02c645-04ed-4160-ac23-419ffa0d50e0 None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:05:49.790698 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d02c645-04ed-4160-ac23-419ffa0d50e0 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:05:49.791130 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d02c645-04ed-4160-ac23-419ffa0d50e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:05:49.803048 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1016/2018] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 11:05:49 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 11:05:49.807956 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90b568e-1362-42e3-8d05-9ed510ad88cc None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:05:49.809498 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-b90b568e-1362-42e3-8d05-9ed510ad88cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:05:49.815137 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-8d02c645-04ed-4160-ac23-419ffa0d50e0 None None] MFA Rules not processed for user `ca155a48e2024c32af3fd2e19e5fac11`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 11:05:49.816331 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-8d02c645-04ed-4160-ac23-419ffa0d50e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:05:49.824412 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90b568e-1362-42e3-8d05-9ed510ad88cc None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:05:49.887546 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1003/2019] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 11:05:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 11:05:49.895374 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8537e52-77cf-45cb-b85e-11a8b3be2cc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:05:49.895531 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63b8698-7aa1-4db8-bb0f-33ceee48b987 None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:05:49.895759 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63b8698-7aa1-4db8-bb0f-33ceee48b987 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:05:49.895994 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8537e52-77cf-45cb-b85e-11a8b3be2cc4 admin admin] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:05:49.896142 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63b8698-7aa1-4db8-bb0f-33ceee48b987 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:05:49.896282 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8537e52-77cf-45cb-b85e-11a8b3be2cc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:05:49.896555 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8537e52-77cf-45cb-b85e-11a8b3be2cc4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:05:49.897190 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8537e52-77cf-45cb-b85e-11a8b3be2cc4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:05:49.899227 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8537e52-77cf-45cb-b85e-11a8b3be2cc4 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:05:49.916201 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-b63b8698-7aa1-4db8-bb0f-33ceee48b987 None None] MFA Rules not processed for user `ca155a48e2024c32af3fd2e19e5fac11`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 11:05:49.917773 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-b63b8698-7aa1-4db8-bb0f-33ceee48b987 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:05:49.926376 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1004/2020] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 11:05:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 11:05:49.931235 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63b8698-7aa1-4db8-bb0f-33ceee48b987 None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:05:49.931725 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1017/2021] 10.222.0.40 () {60 vars in 1153 bytes} [Fri Aug 26 11:05:49 2022] POST /identity/v3/projects => generated 371 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 11:05:49.935241 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8537e52-77cf-45cb-b85e-11a8b3be2cc4 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:05:49.940803 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63b8698-7aa1-4db8-bb0f-33ceee48b987 None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:05:49.986634 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:DELETE=3,keystone:SELECT=95,keystone:INSERT=2 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 11:05:49.997015 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bfb700f-7f09-411e-8023-5c6e8b19f6f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:05:49.997584 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bfb700f-7f09-411e-8023-5c6e8b19f6f3 admin admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:05:49.997866 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bfb700f-7f09-411e-8023-5c6e8b19f6f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:05:49.998174 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bfb700f-7f09-411e-8023-5c6e8b19f6f3 admin admin] PATH_INFO: `/v3/domains` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:05:49.998792 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8537e52-77cf-45cb-b85e-11a8b3be2cc4 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:05:49.998889 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bfb700f-7f09-411e-8023-5c6e8b19f6f3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 11:05:49.999113 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bfb700f-7f09-411e-8023-5c6e8b19f6f3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:05:50.000744 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bfb700f-7f09-411e-8023-5c6e8b19f6f3 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:05:50.004763 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c75478f-1190-4cb7-9b97-d31ad7b52ee8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:05:50.005387 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c75478f-1190-4cb7-9b97-d31ad7b52ee8 admin admin] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:05:50.005885 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c75478f-1190-4cb7-9b97-d31ad7b52ee8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:05:50.006194 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c75478f-1190-4cb7-9b97-d31ad7b52ee8 admin admin] PATH_INFO: `/v3/users` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:05:50.006826 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c75478f-1190-4cb7-9b97-d31ad7b52ee8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:05:50.008771 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c75478f-1190-4cb7-9b97-d31ad7b52ee8 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:05:50.014063 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1005/2022] 10.222.0.40 () {58 vars in 1153 bytes} [Fri Aug 26 11:05:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 11:05:50.018762 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bfb700f-7f09-411e-8023-5c6e8b19f6f3 admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:05:50.030899 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bfb700f-7f09-411e-8023-5c6e8b19f6f3 admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:05:50.033555 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1018/2023] 10.222.0.40 () {60 vars in 1144 bytes} [Fri Aug 26 11:05:49 2022] POST /identity/v3/users => generated 292 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 26 11:05:50.035944 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf36dae7-8ef8-417a-b4c8-0ff6429a3154 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:05:50.036573 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf36dae7-8ef8-417a-b4c8-0ff6429a3154 admin admin] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:05:50.037388 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf36dae7-8ef8-417a-b4c8-0ff6429a3154 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:05:50.037644 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c75478f-1190-4cb7-9b97-d31ad7b52ee8 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:05:50.037811 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf36dae7-8ef8-417a-b4c8-0ff6429a3154 admin admin] PATH_INFO: `/v3/projects` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:05:50.038633 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf36dae7-8ef8-417a-b4c8-0ff6429a3154 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:05:50.040649 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf36dae7-8ef8-417a-b4c8-0ff6429a3154 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:05:50.049381 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c75478f-1190-4cb7-9b97-d31ad7b52ee8 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:05:50.056053 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7655b1-b00f-4ab1-adb2-ccfe7287f741 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:05:50.056668 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7655b1-b00f-4ab1-adb2-ccfe7287f741 admin admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:05:50.056952 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7655b1-b00f-4ab1-adb2-ccfe7287f741 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:05:50.057223 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7655b1-b00f-4ab1-adb2-ccfe7287f741 admin admin] PATH_INFO: `/v3/roles` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:05:50.057877 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d7655b1-b00f-4ab1-adb2-ccfe7287f741 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:05:50.059501 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d7655b1-b00f-4ab1-adb2-ccfe7287f741 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:05:50.071386 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1019/2024] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 26 11:05:50 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 26 11:05:50.076729 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7655b1-b00f-4ab1-adb2-ccfe7287f741 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:05:50.078745 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1006/2025] 10.222.0.40 () {60 vars in 1153 bytes} [Fri Aug 26 11:05:50 2022] POST /identity/v3/projects => generated 389 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 26 11:05:50.082628 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf36dae7-8ef8-417a-b4c8-0ff6429a3154 admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:05:50.086874 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7655b1-b00f-4ab1-adb2-ccfe7287f741 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:05:50.118526 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:DELETE=5,keystone:SELECT=93,keystone:INSERT=2 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 11:05:50.145280 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dbb99c9-defb-4960-8161-61d509649ade admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:05:50.145951 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dbb99c9-defb-4960-8161-61d509649ade admin admin] REQUEST_METHOD: `PUT` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:05:50.146843 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dbb99c9-defb-4960-8161-61d509649ade admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:05:50.147234 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dbb99c9-defb-4960-8161-61d509649ade admin admin] PATH_INFO: `/v3/projects/57a6b40b9bcb4840851fb48cb9a19f6d/users/5110464c90a64882b18292c370097798/roles/1f97e4602e2e4d6498913a81f9ae8d6a` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:05:50.153155 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf36dae7-8ef8-417a-b4c8-0ff6429a3154 admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:05:50.158936 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0222b576-412b-4834-84b6-7c714b9d3b93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:05:50.159549 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0222b576-412b-4834-84b6-7c714b9d3b93 admin admin] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:05:50.159848 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0222b576-412b-4834-84b6-7c714b9d3b93 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:05:50.160099 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0222b576-412b-4834-84b6-7c714b9d3b93 admin admin] PATH_INFO: `/v3/users` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:05:50.160741 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0222b576-412b-4834-84b6-7c714b9d3b93 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:05:50.161464 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dbb99c9-defb-4960-8161-61d509649ade admin admin] RBAC: Authorizing `identity:create_grant(project_id=57a6b40b9bcb4840851fb48cb9a19f6d, user_id=5110464c90a64882b18292c370097798, role_id=1f97e4602e2e4d6498913a81f9ae8d6a)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:05:50.163461 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0222b576-412b-4834-84b6-7c714b9d3b93 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:05:50.164713 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dbb99c9-defb-4960-8161-61d509649ade admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:05:50.179170 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1020/2026] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 26 11:05:50 2022] PUT /identity/v3/projects/57a6b40b9bcb4840851fb48cb9a19f6d/users/5110464c90a64882b18292c370097798/roles/1f97e4602e2e4d6498913a81f9ae8d6a => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 26 11:05:50.184495 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dbb99c9-defb-4960-8161-61d509649ade admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:05:50.189120 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1007/2027] 10.222.0.40 () {60 vars in 1144 bytes} [Fri Aug 26 11:05:50 2022] POST /identity/v3/users => generated 295 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 11:05:50.193202 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0222b576-412b-4834-84b6-7c714b9d3b93 admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:05:50.203075 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0222b576-412b-4834-84b6-7c714b9d3b93 admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:05:50.248982 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dbb99c9-defb-4960-8161-61d509649ade admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:05:50.253706 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a17ff3-bfeb-40b1-9e87-db728219560d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:05:50.254343 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a17ff3-bfeb-40b1-9e87-db728219560d admin admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:05:50.254624 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a17ff3-bfeb-40b1-9e87-db728219560d admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:05:50.254891 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a17ff3-bfeb-40b1-9e87-db728219560d admin admin] PATH_INFO: `/v3/roles` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:05:50.255620 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a17ff3-bfeb-40b1-9e87-db728219560d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:05:50.258338 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a17ff3-bfeb-40b1-9e87-db728219560d admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:05:50.268487 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1021/2028] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 26 11:05:50 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 26 11:05:50.269800 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171864fa-9dc1-4730-9b46-9481333217ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:05:50.270450 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171864fa-9dc1-4730-9b46-9481333217ae admin admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:05:50.270733 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171864fa-9dc1-4730-9b46-9481333217ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:05:50.271007 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171864fa-9dc1-4730-9b46-9481333217ae admin admin] PATH_INFO: `/v3/roles` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:05:50.272186 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-171864fa-9dc1-4730-9b46-9481333217ae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:05:50.273191 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a17ff3-bfeb-40b1-9e87-db728219560d admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:05:50.273776 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-171864fa-9dc1-4730-9b46-9481333217ae admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:05:50.282822 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a17ff3-bfeb-40b1-9e87-db728219560d admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:05:50.284095 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1008/2029] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 26 11:05:50 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 11:05:50.288693 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171864fa-9dc1-4730-9b46-9481333217ae admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:05:50.288844 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7067630-2c98-490b-a1b3-b6eea38f11c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:05:50.289502 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7067630-2c98-490b-a1b3-b6eea38f11c2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:05:50.289771 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7067630-2c98-490b-a1b3-b6eea38f11c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:05:50.290222 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7067630-2c98-490b-a1b3-b6eea38f11c2 admin admin] PATH_INFO: `/v3/projects/57a6b40b9bcb4840851fb48cb9a19f6d/users/5110464c90a64882b18292c370097798/roles/1f97e4602e2e4d6498913a81f9ae8d6a` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:05:50.292881 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7067630-2c98-490b-a1b3-b6eea38f11c2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=57a6b40b9bcb4840851fb48cb9a19f6d, user_id=5110464c90a64882b18292c370097798, role_id=1f97e4602e2e4d6498913a81f9ae8d6a)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:05:50.295922 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7067630-2c98-490b-a1b3-b6eea38f11c2 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:05:50.299074 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171864fa-9dc1-4730-9b46-9481333217ae admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:05:50.303939 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c539c9-6bff-46c3-8270-7008a1ff4e7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:05:50.304535 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c539c9-6bff-46c3-8270-7008a1ff4e7e admin admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:05:50.304815 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c539c9-6bff-46c3-8270-7008a1ff4e7e admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:05:50.305090 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c539c9-6bff-46c3-8270-7008a1ff4e7e admin admin] PATH_INFO: `/v3/roles` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:05:50.305790 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c539c9-6bff-46c3-8270-7008a1ff4e7e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:05:50.307358 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c539c9-6bff-46c3-8270-7008a1ff4e7e admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:05:50.307730 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1022/2030] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 26 11:05:50 2022] PUT /identity/v3/projects/57a6b40b9bcb4840851fb48cb9a19f6d/users/5110464c90a64882b18292c370097798/roles/1f97e4602e2e4d6498913a81f9ae8d6a => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 11:05:50.321165 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1009/2031] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 26 11:05:50 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 11:05:50.322348 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7067630-2c98-490b-a1b3-b6eea38f11c2 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:05:50.371621 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 11:05:50.399563 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7067630-2c98-490b-a1b3-b6eea38f11c2 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:05:50.404676 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b5f6d7-5c24-4d5e-a526-a65afe784bda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:05:50.405565 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b5f6d7-5c24-4d5e-a526-a65afe784bda admin admin] REQUEST_METHOD: `PUT` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:05:50.405839 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b5f6d7-5c24-4d5e-a526-a65afe784bda admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:05:50.406168 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b5f6d7-5c24-4d5e-a526-a65afe784bda admin admin] PATH_INFO: `/v3/projects/7025fe0928ec4955b13c1f185e72f9e0/users/949a4c56bc6548ad9eb32e54d1f43ae7/roles/23e174ce743b46eabe618c212691da55` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:05:50.424057 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b5f6d7-5c24-4d5e-a526-a65afe784bda admin admin] RBAC: Authorizing `identity:create_grant(project_id=7025fe0928ec4955b13c1f185e72f9e0, user_id=949a4c56bc6548ad9eb32e54d1f43ae7, role_id=23e174ce743b46eabe618c212691da55)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:05:50.426887 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b5f6d7-5c24-4d5e-a526-a65afe784bda admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:05:50.440375 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1023/2032] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 26 11:05:50 2022] PUT /identity/v3/projects/7025fe0928ec4955b13c1f185e72f9e0/users/949a4c56bc6548ad9eb32e54d1f43ae7/roles/23e174ce743b46eabe618c212691da55 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 26 11:05:50.452030 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb57e82c-b188-4ee7-b4c6-430c6c54a60e None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:05:50.452305 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb57e82c-b188-4ee7-b4c6-430c6c54a60e None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:05:50.452576 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb57e82c-b188-4ee7-b4c6-430c6c54a60e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:05:50.483935 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-cb57e82c-b188-4ee7-b4c6-430c6c54a60e None None] MFA Rules not processed for user `949a4c56bc6548ad9eb32e54d1f43ae7`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 11:05:50.485284 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-cb57e82c-b188-4ee7-b4c6-430c6c54a60e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:05:50.562735 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1010/2033] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 11:05:50 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 11:05:50.570999 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b5f6d7-5c24-4d5e-a526-a65afe784bda admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:05:50.573239 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-e8b5f6d7-5c24-4d5e-a526-a65afe784bda admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:05:50.641686 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b5f6d7-5c24-4d5e-a526-a65afe784bda admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:05:50.645427 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b02f83-83bf-4152-aece-f5ad251cbd3b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c277338a7f814cc6b01c5419775031bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e30ccf0c1204727b786876c141dbcd2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:05:50.646194 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b02f83-83bf-4152-aece-f5ad251cbd3b service cinder] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:05:50.647218 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b02f83-83bf-4152-aece-f5ad251cbd3b service cinder] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:05:50.647728 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b02f83-83bf-4152-aece-f5ad251cbd3b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:05:50.657092 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b02f83-83bf-4152-aece-f5ad251cbd3b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:05:50.658645 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b02f83-83bf-4152-aece-f5ad251cbd3b service cinder] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:05:50.668218 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1024/2034] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 11:05:50 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 11:05:51.076062 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8199459-4758-468e-a148-30ca6b90b502 None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:05:51.076815 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8199459-4758-468e-a148-30ca6b90b502 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:05:51.076815 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8199459-4758-468e-a148-30ca6b90b502 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:05:51.108839 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-b8199459-4758-468e-a148-30ca6b90b502 None None] MFA Rules not processed for user `5110464c90a64882b18292c370097798`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 11:05:51.110088 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-b8199459-4758-468e-a148-30ca6b90b502 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:05:51.136598 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 11:05:51.194456 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1011/2035] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 11:05:51 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 11:05:51.205599 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b02f83-83bf-4152-aece-f5ad251cbd3b service cinder] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:05:51.214948 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b02f83-83bf-4152-aece-f5ad251cbd3b service cinder] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:05:51.218969 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7a670e-b7fb-41bd-8059-3e774b694001 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c277338a7f814cc6b01c5419775031bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e30ccf0c1204727b786876c141dbcd2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:05:51.219754 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7a670e-b7fb-41bd-8059-3e774b694001 service cinder] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:05:51.220037 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7a670e-b7fb-41bd-8059-3e774b694001 service cinder] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:05:51.220382 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7a670e-b7fb-41bd-8059-3e774b694001 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:05:51.279463 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e7a670e-b7fb-41bd-8059-3e774b694001 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:05:51.280833 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e7a670e-b7fb-41bd-8059-3e774b694001 service cinder] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:05:51.290383 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1025/2036] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 11:05:51 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 11:05:53.723412 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8199459-4758-468e-a148-30ca6b90b502 None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:05:53.734937 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8199459-4758-468e-a148-30ca6b90b502 None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:05:53.739093 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2002be76-4eb3-4039-9f38-af94c9b35219 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c277338a7f814cc6b01c5419775031bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e30ccf0c1204727b786876c141dbcd2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:05:53.739721 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2002be76-4eb3-4039-9f38-af94c9b35219 service cinder] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:05:53.740649 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2002be76-4eb3-4039-9f38-af94c9b35219 service cinder] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:05:53.740930 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2002be76-4eb3-4039-9f38-af94c9b35219 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:05:53.752193 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2002be76-4eb3-4039-9f38-af94c9b35219 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:05:53.753632 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2002be76-4eb3-4039-9f38-af94c9b35219 service cinder] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:05:53.772300 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1012/2037] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 11:05:53 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 11:05:59.497523 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7a670e-b7fb-41bd-8059-3e774b694001 service cinder] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:05:59.499189 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-3e7a670e-b7fb-41bd-8059-3e774b694001 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:05:59.595921 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7a670e-b7fb-41bd-8059-3e774b694001 service cinder] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:05:59.602109 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258ed7ad-2cb6-48b5-abdc-731532b43465 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:05:59.602752 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258ed7ad-2cb6-48b5-abdc-731532b43465 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:05:59.603033 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258ed7ad-2cb6-48b5-abdc-731532b43465 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:05:59.603304 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258ed7ad-2cb6-48b5-abdc-731532b43465 admin admin] PATH_INFO: `/v3/users/fd154caf7c5d49e59e916666eca681b6` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:05:59.604605 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-258ed7ad-2cb6-48b5-abdc-731532b43465 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fd154caf7c5d49e59e916666eca681b6)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:05:59.606572 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-258ed7ad-2cb6-48b5-abdc-731532b43465 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:05:59.609650 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 11:05:59.671951 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-258ed7ad-2cb6-48b5-abdc-731532b43465 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fd154caf7c5d49e59e916666eca681b6'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:05:59.672661 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-258ed7ad-2cb6-48b5-abdc-731532b43465 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fd154caf7c5d49e59e916666eca681b6'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:05:59.680343 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-258ed7ad-2cb6-48b5-abdc-731532b43465 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fd154caf7c5d49e59e916666eca681b6'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:05:59.681004 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-258ed7ad-2cb6-48b5-abdc-731532b43465 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fd154caf7c5d49e59e916666eca681b6'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:05:59.694260 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-258ed7ad-2cb6-48b5-abdc-731532b43465 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fd154caf7c5d49e59e916666eca681b6'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:05:59.707743 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1026/2038] 10.222.0.40 () {58 vars in 1224 bytes} [Fri Aug 26 11:05:59 2022] DELETE /identity/v3/users/fd154caf7c5d49e59e916666eca681b6 => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 11:05:59.712569 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d18bfa-19cf-4ff7-b9d7-3a7d5da3a222 None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:05:59.713022 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d18bfa-19cf-4ff7-b9d7-3a7d5da3a222 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:05:59.713154 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d18bfa-19cf-4ff7-b9d7-3a7d5da3a222 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:05:59.734078 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-11d18bfa-19cf-4ff7-b9d7-3a7d5da3a222 None None] MFA Rules not processed for user `ca155a48e2024c32af3fd2e19e5fac11`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 11:05:59.735298 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-11d18bfa-19cf-4ff7-b9d7-3a7d5da3a222 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:05:59.799189 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1013/2039] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 11:05:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 11:05:59.809691 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258ed7ad-2cb6-48b5-abdc-731532b43465 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:05:59.817205 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-258ed7ad-2cb6-48b5-abdc-731532b43465 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:05:59.876349 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258ed7ad-2cb6-48b5-abdc-731532b43465 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:05:59.880833 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1241d80-2c51-4e13-980a-8da000df389d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91da2acd2534423091537e1ba068d2b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e30ccf0c1204727b786876c141dbcd2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:05:59.881470 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1241d80-2c51-4e13-980a-8da000df389d service neutron] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:05:59.881743 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1241d80-2c51-4e13-980a-8da000df389d service neutron] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:05:59.882043 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1241d80-2c51-4e13-980a-8da000df389d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:05:59.891399 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1241d80-2c51-4e13-980a-8da000df389d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:05:59.892808 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1241d80-2c51-4e13-980a-8da000df389d service neutron] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:05:59.905428 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1027/2040] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 11:05:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 11:06:00.206751 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d18bfa-19cf-4ff7-b9d7-3a7d5da3a222 None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:06:00.208196 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-11d18bfa-19cf-4ff7-b9d7-3a7d5da3a222 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:06:00.225531 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d18bfa-19cf-4ff7-b9d7-3a7d5da3a222 None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:06:00.230324 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf8914b-1c03-4c5b-a08d-29a05bc3fa15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:06:00.230971 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf8914b-1c03-4c5b-a08d-29a05bc3fa15 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:06:00.231287 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf8914b-1c03-4c5b-a08d-29a05bc3fa15 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:06:00.231502 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf8914b-1c03-4c5b-a08d-29a05bc3fa15 admin admin] PATH_INFO: `/v3/projects/8c339a176ac445689940c78db39d1370` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:06:00.232680 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf8914b-1c03-4c5b-a08d-29a05bc3fa15 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8c339a176ac445689940c78db39d1370)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:06:00.234528 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf8914b-1c03-4c5b-a08d-29a05bc3fa15 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:06:00.285463 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-caf8914b-1c03-4c5b-a08d-29a05bc3fa15 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8c339a176ac445689940c78db39d1370', 'request_id': 'req-caf8914b-1c03-4c5b-a08d-29a05bc3fa15'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:06:00.293657 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-caf8914b-1c03-4c5b-a08d-29a05bc3fa15 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8c339a176ac445689940c78db39d1370', 'request_id': 'req-caf8914b-1c03-4c5b-a08d-29a05bc3fa15'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:06:00.301773 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-caf8914b-1c03-4c5b-a08d-29a05bc3fa15 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8c339a176ac445689940c78db39d1370', 'request_id': 'req-caf8914b-1c03-4c5b-a08d-29a05bc3fa15'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:06:00.314614 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-caf8914b-1c03-4c5b-a08d-29a05bc3fa15 admin admin] The token cache is being invalidate because project 8c339a176ac445689940c78db39d1370 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81423) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 11:06:00.314904 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-caf8914b-1c03-4c5b-a08d-29a05bc3fa15 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:06:00.315549 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-caf8914b-1c03-4c5b-a08d-29a05bc3fa15 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:06:00.317075 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1014/2041] 10.222.0.40 () {58 vars in 1233 bytes} [Fri Aug 26 11:06:00 2022] DELETE /identity/v3/projects/8c339a176ac445689940c78db39d1370 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 26 11:06:00.321877 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1241d80-2c51-4e13-980a-8da000df389d service neutron] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:06:00.332341 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1241d80-2c51-4e13-980a-8da000df389d service neutron] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:06:00.389692 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1fe1e0-c021-4eda-9949-4592fd2d2ab3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:06:00.390342 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1fe1e0-c021-4eda-9949-4592fd2d2ab3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:06:00.390617 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1fe1e0-c021-4eda-9949-4592fd2d2ab3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:06:00.390887 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1fe1e0-c021-4eda-9949-4592fd2d2ab3 admin admin] PATH_INFO: `/v3/users/153feb43d9f346ed896b097bd4132526` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:06:00.392021 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae1fe1e0-c021-4eda-9949-4592fd2d2ab3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=153feb43d9f346ed896b097bd4132526)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:06:00.393917 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae1fe1e0-c021-4eda-9949-4592fd2d2ab3 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:06:00.421721 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 11:06:00.441398 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-ae1fe1e0-c021-4eda-9949-4592fd2d2ab3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '153feb43d9f346ed896b097bd4132526'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:06:00.442072 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-ae1fe1e0-c021-4eda-9949-4592fd2d2ab3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '153feb43d9f346ed896b097bd4132526'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:06:00.449523 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-ae1fe1e0-c021-4eda-9949-4592fd2d2ab3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '153feb43d9f346ed896b097bd4132526'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:06:00.450194 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-ae1fe1e0-c021-4eda-9949-4592fd2d2ab3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '153feb43d9f346ed896b097bd4132526'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:06:00.462634 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-ae1fe1e0-c021-4eda-9949-4592fd2d2ab3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '153feb43d9f346ed896b097bd4132526'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:06:00.474814 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1028/2042] 10.222.0.40 () {58 vars in 1224 bytes} [Fri Aug 26 11:06:00 2022] DELETE /identity/v3/users/153feb43d9f346ed896b097bd4132526 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 26 11:06:00.748422 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf8914b-1c03-4c5b-a08d-29a05bc3fa15 admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:06:00.749871 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-caf8914b-1c03-4c5b-a08d-29a05bc3fa15 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:06:00.768168 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf8914b-1c03-4c5b-a08d-29a05bc3fa15 admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:06:00.801120 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 11:06:00.834154 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06823c17-be8f-407e-a2a8-f85be11f5064 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:06:00.834798 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06823c17-be8f-407e-a2a8-f85be11f5064 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:06:00.835076 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06823c17-be8f-407e-a2a8-f85be11f5064 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:06:00.835606 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06823c17-be8f-407e-a2a8-f85be11f5064 admin admin] PATH_INFO: `/v3/projects/2c5c7ed5385b4a2dafe1b0f3c4601f64` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:06:00.836757 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06823c17-be8f-407e-a2a8-f85be11f5064 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2c5c7ed5385b4a2dafe1b0f3c4601f64)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:06:00.838715 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06823c17-be8f-407e-a2a8-f85be11f5064 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:06:00.895535 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-06823c17-be8f-407e-a2a8-f85be11f5064 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2c5c7ed5385b4a2dafe1b0f3c4601f64', 'request_id': 'req-06823c17-be8f-407e-a2a8-f85be11f5064'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:06:00.902240 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-06823c17-be8f-407e-a2a8-f85be11f5064 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2c5c7ed5385b4a2dafe1b0f3c4601f64', 'request_id': 'req-06823c17-be8f-407e-a2a8-f85be11f5064'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:06:00.907894 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-06823c17-be8f-407e-a2a8-f85be11f5064 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2c5c7ed5385b4a2dafe1b0f3c4601f64', 'request_id': 'req-06823c17-be8f-407e-a2a8-f85be11f5064'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:06:00.920761 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-06823c17-be8f-407e-a2a8-f85be11f5064 admin admin] The token cache is being invalidate because project 2c5c7ed5385b4a2dafe1b0f3c4601f64 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81423) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 11:06:00.921044 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-06823c17-be8f-407e-a2a8-f85be11f5064 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:06:00.921694 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-06823c17-be8f-407e-a2a8-f85be11f5064 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:06:00.923233 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1015/2043] 10.222.0.40 () {58 vars in 1233 bytes} [Fri Aug 26 11:06:00 2022] DELETE /identity/v3/projects/2c5c7ed5385b4a2dafe1b0f3c4601f64 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 11:06:00.930252 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a01cc49-1c36-48c6-abb0-7a42052112ba None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:06:00.930757 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a01cc49-1c36-48c6-abb0-7a42052112ba None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:06:00.930849 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a01cc49-1c36-48c6-abb0-7a42052112ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:06:00.950746 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-0a01cc49-1c36-48c6-abb0-7a42052112ba None None] MFA Rules not processed for user `ca155a48e2024c32af3fd2e19e5fac11`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 11:06:00.951968 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-0a01cc49-1c36-48c6-abb0-7a42052112ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:06:01.010473 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1029/2044] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 11:06:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 11:06:01.018033 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3752f703-7cfe-4bc4-9ed2-d7551e24971c None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:06:01.018318 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3752f703-7cfe-4bc4-9ed2-d7551e24971c None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:06:01.018610 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3752f703-7cfe-4bc4-9ed2-d7551e24971c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:06:01.038608 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-3752f703-7cfe-4bc4-9ed2-d7551e24971c None None] MFA Rules not processed for user `ca155a48e2024c32af3fd2e19e5fac11`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 11:06:01.039795 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-3752f703-7cfe-4bc4-9ed2-d7551e24971c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:06:01.047653 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1016/2045] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 11:06:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 11:06:01.051937 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a01cc49-1c36-48c6-abb0-7a42052112ba None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:06:01.062889 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a01cc49-1c36-48c6-abb0-7a42052112ba None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:06:01.067557 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c61c8e-8663-44e7-a9a0-d220988db424 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:06:01.068443 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c61c8e-8663-44e7-a9a0-d220988db424 admin admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:06:01.068717 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c61c8e-8663-44e7-a9a0-d220988db424 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:06:01.068986 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c61c8e-8663-44e7-a9a0-d220988db424 admin admin] PATH_INFO: `/v3/domains` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:06:01.069635 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c61c8e-8663-44e7-a9a0-d220988db424 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 11:06:01.069934 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c61c8e-8663-44e7-a9a0-d220988db424 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:06:01.071538 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c61c8e-8663-44e7-a9a0-d220988db424 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:06:01.084808 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1030/2046] 10.222.0.40 () {58 vars in 1153 bytes} [Fri Aug 26 11:06:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 11:06:01.089226 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3752f703-7cfe-4bc4-9ed2-d7551e24971c None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:06:01.098530 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3752f703-7cfe-4bc4-9ed2-d7551e24971c None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:06:01.104033 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9140e84a-5cd1-49a5-bbd3-1c887ca7dab8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:06:01.104629 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9140e84a-5cd1-49a5-bbd3-1c887ca7dab8 admin admin] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:06:01.104902 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9140e84a-5cd1-49a5-bbd3-1c887ca7dab8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:06:01.105167 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9140e84a-5cd1-49a5-bbd3-1c887ca7dab8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:06:01.105797 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9140e84a-5cd1-49a5-bbd3-1c887ca7dab8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:06:01.107714 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9140e84a-5cd1-49a5-bbd3-1c887ca7dab8 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:06:01.139511 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1017/2047] 10.222.0.40 () {60 vars in 1153 bytes} [Fri Aug 26 11:06:01 2022] POST /identity/v3/projects => generated 363 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 11:06:01.143871 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c61c8e-8663-44e7-a9a0-d220988db424 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:06:01.158548 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c61c8e-8663-44e7-a9a0-d220988db424 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:06:01.226471 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05b64ad-a35d-4482-9229-9d3406c46e84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:06:01.227093 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05b64ad-a35d-4482-9229-9d3406c46e84 admin admin] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:06:01.227369 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05b64ad-a35d-4482-9229-9d3406c46e84 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:06:01.227637 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05b64ad-a35d-4482-9229-9d3406c46e84 admin admin] PATH_INFO: `/v3/users` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:06:01.228271 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a05b64ad-a35d-4482-9229-9d3406c46e84 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:06:01.230252 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a05b64ad-a35d-4482-9229-9d3406c46e84 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:06:01.253131 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1031/2048] 10.222.0.40 () {60 vars in 1144 bytes} [Fri Aug 26 11:06:01 2022] POST /identity/v3/users => generated 282 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 11:06:01.258125 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9140e84a-5cd1-49a5-bbd3-1c887ca7dab8 admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:06:01.268822 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9140e84a-5cd1-49a5-bbd3-1c887ca7dab8 admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:06:01.273640 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66bac7b2-9c6d-4348-bb8c-805ac605c990 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:06:01.274271 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66bac7b2-9c6d-4348-bb8c-805ac605c990 admin admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:06:01.274552 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66bac7b2-9c6d-4348-bb8c-805ac605c990 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:06:01.274823 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66bac7b2-9c6d-4348-bb8c-805ac605c990 admin admin] PATH_INFO: `/v3/roles` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:06:01.275458 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66bac7b2-9c6d-4348-bb8c-805ac605c990 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:06:01.277040 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66bac7b2-9c6d-4348-bb8c-805ac605c990 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:06:01.288325 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1018/2049] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 26 11:06:01 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 11:06:01.292666 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05b64ad-a35d-4482-9229-9d3406c46e84 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:06:01.303789 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05b64ad-a35d-4482-9229-9d3406c46e84 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:06:01.308494 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322ebd82-7e1e-43d6-8712-8c390341794a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:06:01.309078 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322ebd82-7e1e-43d6-8712-8c390341794a admin admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:06:01.309351 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322ebd82-7e1e-43d6-8712-8c390341794a admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:06:01.309618 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322ebd82-7e1e-43d6-8712-8c390341794a admin admin] PATH_INFO: `/v3/roles` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:06:01.310276 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-322ebd82-7e1e-43d6-8712-8c390341794a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:06:01.311849 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-322ebd82-7e1e-43d6-8712-8c390341794a admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:06:01.322372 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1032/2050] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 26 11:06:01 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 11:06:01.326503 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66bac7b2-9c6d-4348-bb8c-805ac605c990 admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:06:01.336531 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66bac7b2-9c6d-4348-bb8c-805ac605c990 admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:06:01.341242 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3ca830-499a-4c8c-a87f-b9a7370e9ea5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:06:01.341889 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3ca830-499a-4c8c-a87f-b9a7370e9ea5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:06:01.342268 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3ca830-499a-4c8c-a87f-b9a7370e9ea5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:06:01.342544 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3ca830-499a-4c8c-a87f-b9a7370e9ea5 admin admin] PATH_INFO: `/v3/projects/ecdd6d54de5e47df9a0ecaa4c836b7fc/users/2e03effab5f54b648287cb93cff0a8a0/roles/23e174ce743b46eabe618c212691da55` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:06:01.358747 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a3ca830-499a-4c8c-a87f-b9a7370e9ea5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ecdd6d54de5e47df9a0ecaa4c836b7fc, user_id=2e03effab5f54b648287cb93cff0a8a0, role_id=23e174ce743b46eabe618c212691da55)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:06:01.361357 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a3ca830-499a-4c8c-a87f-b9a7370e9ea5 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:06:01.376150 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1019/2051] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 26 11:06:01 2022] PUT /identity/v3/projects/ecdd6d54de5e47df9a0ecaa4c836b7fc/users/2e03effab5f54b648287cb93cff0a8a0/roles/23e174ce743b46eabe618c212691da55 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 11:06:01.386962 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9252313a-133a-4e41-988e-7208eabebdc7 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:06:01.387318 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9252313a-133a-4e41-988e-7208eabebdc7 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:06:01.387654 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9252313a-133a-4e41-988e-7208eabebdc7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:06:01.409019 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 11:06:01.420981 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-9252313a-133a-4e41-988e-7208eabebdc7 None None] MFA Rules not processed for user `2e03effab5f54b648287cb93cff0a8a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 11:06:01.422610 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-9252313a-133a-4e41-988e-7208eabebdc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:06:01.487157 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1033/2052] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 11:06:01 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 11:06:01.491826 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3ca830-499a-4c8c-a87f-b9a7370e9ea5 admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:06:01.548837 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 11:06:01.555137 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3ca830-499a-4c8c-a87f-b9a7370e9ea5 admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:06:01.560676 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52bfe54-01ce-4cb5-998e-1ff0309ab187 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:06:01.561274 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52bfe54-01ce-4cb5-998e-1ff0309ab187 admin admin] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:06:01.561553 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52bfe54-01ce-4cb5-998e-1ff0309ab187 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:06:01.561829 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52bfe54-01ce-4cb5-998e-1ff0309ab187 admin admin] PATH_INFO: `/v3/projects` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:06:01.562492 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e52bfe54-01ce-4cb5-998e-1ff0309ab187 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:06:01.564388 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e52bfe54-01ce-4cb5-998e-1ff0309ab187 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:06:01.590751 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1020/2053] 10.222.0.40 () {60 vars in 1153 bytes} [Fri Aug 26 11:06:01 2022] POST /identity/v3/projects => generated 365 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 26 11:06:01.594817 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9252313a-133a-4e41-988e-7208eabebdc7 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:06:01.606155 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9252313a-133a-4e41-988e-7208eabebdc7 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:06:01.660629 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44b492a-d7f5-4e84-a084-268345e66759 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:06:01.661225 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44b492a-d7f5-4e84-a084-268345e66759 admin admin] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:06:01.661498 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44b492a-d7f5-4e84-a084-268345e66759 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:06:01.662418 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44b492a-d7f5-4e84-a084-268345e66759 admin admin] PATH_INFO: `/v3/users` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:06:01.663024 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c44b492a-d7f5-4e84-a084-268345e66759 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:06:01.664918 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c44b492a-d7f5-4e84-a084-268345e66759 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:06:01.685753 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1034/2054] 10.222.0.40 () {60 vars in 1144 bytes} [Fri Aug 26 11:06:01 2022] POST /identity/v3/users => generated 289 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 11:06:01.689694 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52bfe54-01ce-4cb5-998e-1ff0309ab187 admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:06:01.700101 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52bfe54-01ce-4cb5-998e-1ff0309ab187 admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:06:01.704804 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3718cb3d-7d2d-4697-85bd-173b1c15a5f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:06:01.705415 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3718cb3d-7d2d-4697-85bd-173b1c15a5f4 admin admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:06:01.705684 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3718cb3d-7d2d-4697-85bd-173b1c15a5f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:06:01.705961 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3718cb3d-7d2d-4697-85bd-173b1c15a5f4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:06:01.706651 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3718cb3d-7d2d-4697-85bd-173b1c15a5f4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:06:01.708234 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3718cb3d-7d2d-4697-85bd-173b1c15a5f4 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:06:01.719369 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1021/2055] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 26 11:06:01 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 11:06:01.723602 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44b492a-d7f5-4e84-a084-268345e66759 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:06:01.734214 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44b492a-d7f5-4e84-a084-268345e66759 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:06:01.739038 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad6d791-956c-4dc2-8ff4-52cf5bd33884 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:06:01.739723 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad6d791-956c-4dc2-8ff4-52cf5bd33884 admin admin] REQUEST_METHOD: `PUT` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:06:01.740000 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad6d791-956c-4dc2-8ff4-52cf5bd33884 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:06:01.740275 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad6d791-956c-4dc2-8ff4-52cf5bd33884 admin admin] PATH_INFO: `/v3/projects/245cdfba492848b6a2350a9e0342cce0/users/7e8232f1071241158ac6101404b9bdc9/roles/1f97e4602e2e4d6498913a81f9ae8d6a` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:06:01.755259 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad6d791-956c-4dc2-8ff4-52cf5bd33884 admin admin] RBAC: Authorizing `identity:create_grant(project_id=245cdfba492848b6a2350a9e0342cce0, user_id=7e8232f1071241158ac6101404b9bdc9, role_id=1f97e4602e2e4d6498913a81f9ae8d6a)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:06:01.758471 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad6d791-956c-4dc2-8ff4-52cf5bd33884 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:06:01.771001 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1035/2056] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 26 11:06:01 2022] PUT /identity/v3/projects/245cdfba492848b6a2350a9e0342cce0/users/7e8232f1071241158ac6101404b9bdc9/roles/1f97e4602e2e4d6498913a81f9ae8d6a => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 11:06:01.775565 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3718cb3d-7d2d-4697-85bd-173b1c15a5f4 admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:06:01.785621 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3718cb3d-7d2d-4697-85bd-173b1c15a5f4 admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:06:01.847622 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f3d74aa-affa-4a93-9e68-377d4abda6b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:06:01.848152 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3d74aa-affa-4a93-9e68-377d4abda6b4 admin admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:06:01.848428 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3d74aa-affa-4a93-9e68-377d4abda6b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:06:01.848697 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3d74aa-affa-4a93-9e68-377d4abda6b4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:06:01.849354 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f3d74aa-affa-4a93-9e68-377d4abda6b4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:06:01.851053 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f3d74aa-affa-4a93-9e68-377d4abda6b4 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:06:01.861136 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1022/2057] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 26 11:06:01 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 11:06:01.865410 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad6d791-956c-4dc2-8ff4-52cf5bd33884 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:06:01.876128 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad6d791-956c-4dc2-8ff4-52cf5bd33884 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:06:01.881521 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9b9df1-df85-47e3-9ec5-af12250267d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:06:01.882208 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9b9df1-df85-47e3-9ec5-af12250267d4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:06:01.882481 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9b9df1-df85-47e3-9ec5-af12250267d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:06:01.882750 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9b9df1-df85-47e3-9ec5-af12250267d4 admin admin] PATH_INFO: `/v3/projects/245cdfba492848b6a2350a9e0342cce0/users/7e8232f1071241158ac6101404b9bdc9/roles/1f97e4602e2e4d6498913a81f9ae8d6a` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:06:01.884648 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c9b9df1-df85-47e3-9ec5-af12250267d4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=245cdfba492848b6a2350a9e0342cce0, user_id=7e8232f1071241158ac6101404b9bdc9, role_id=1f97e4602e2e4d6498913a81f9ae8d6a)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:06:01.887325 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c9b9df1-df85-47e3-9ec5-af12250267d4 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:06:01.898626 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1036/2058] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 26 11:06:01 2022] PUT /identity/v3/projects/245cdfba492848b6a2350a9e0342cce0/users/7e8232f1071241158ac6101404b9bdc9/roles/1f97e4602e2e4d6498913a81f9ae8d6a => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 11:06:01.909917 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91757a3e-88cd-43da-aa24-1249a70f6109 None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:06:01.910229 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91757a3e-88cd-43da-aa24-1249a70f6109 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:06:01.910502 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91757a3e-88cd-43da-aa24-1249a70f6109 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:06:01.938187 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-91757a3e-88cd-43da-aa24-1249a70f6109 None None] MFA Rules not processed for user `7e8232f1071241158ac6101404b9bdc9`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 11:06:01.939387 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-91757a3e-88cd-43da-aa24-1249a70f6109 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:06:02.006770 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1023/2059] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 11:06:01 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 11:06:02.018096 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9b9df1-df85-47e3-9ec5-af12250267d4 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:06:02.019810 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-8c9b9df1-df85-47e3-9ec5-af12250267d4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:06:02.075765 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9b9df1-df85-47e3-9ec5-af12250267d4 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:06:02.079441 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfbb868c-a1d8-42b8-aab5-085c6d744509 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c277338a7f814cc6b01c5419775031bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e30ccf0c1204727b786876c141dbcd2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:06:02.080016 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbb868c-a1d8-42b8-aab5-085c6d744509 service cinder] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:06:02.080291 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbb868c-a1d8-42b8-aab5-085c6d744509 service cinder] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:06:02.080561 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbb868c-a1d8-42b8-aab5-085c6d744509 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:06:02.087046 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 11:06:02.091298 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfbb868c-a1d8-42b8-aab5-085c6d744509 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:06:02.092783 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfbb868c-a1d8-42b8-aab5-085c6d744509 service cinder] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:06:02.103134 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1037/2060] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 11:06:02 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 11:06:02.177705 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91757a3e-88cd-43da-aa24-1249a70f6109 None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:06:02.187556 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91757a3e-88cd-43da-aa24-1249a70f6109 None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:06:02.191214 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809fd7f0-645c-486f-90ac-b6beae967d0d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c277338a7f814cc6b01c5419775031bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e30ccf0c1204727b786876c141dbcd2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:06:02.191810 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809fd7f0-645c-486f-90ac-b6beae967d0d service cinder] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:06:02.192095 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809fd7f0-645c-486f-90ac-b6beae967d0d service cinder] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:06:02.192366 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809fd7f0-645c-486f-90ac-b6beae967d0d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:06:02.245871 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 11:06:02.267400 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-809fd7f0-645c-486f-90ac-b6beae967d0d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:06:02.268793 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-809fd7f0-645c-486f-90ac-b6beae967d0d service cinder] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:06:02.281317 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1024/2061] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 11:06:02 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 11:06:03.385353 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfbb868c-a1d8-42b8-aab5-085c6d744509 service cinder] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:06:03.386823 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-cfbb868c-a1d8-42b8-aab5-085c6d744509 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:06:03.451274 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfbb868c-a1d8-42b8-aab5-085c6d744509 service cinder] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:06:03.455929 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f9be5e-c9ed-42b2-a9ed-c14f6cf14666 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:06:03.456544 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f9be5e-c9ed-42b2-a9ed-c14f6cf14666 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:06:03.456821 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f9be5e-c9ed-42b2-a9ed-c14f6cf14666 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:06:03.457092 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f9be5e-c9ed-42b2-a9ed-c14f6cf14666 admin admin] PATH_INFO: `/v3/users/417c2e35727245fbbca6420395e50c44` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:06:03.458287 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f9be5e-c9ed-42b2-a9ed-c14f6cf14666 admin admin] RBAC: Authorizing `identity:delete_user(user_id=417c2e35727245fbbca6420395e50c44)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:06:03.460571 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f9be5e-c9ed-42b2-a9ed-c14f6cf14666 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:06:03.508491 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-c2f9be5e-c9ed-42b2-a9ed-c14f6cf14666 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '417c2e35727245fbbca6420395e50c44'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:06:03.509159 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-c2f9be5e-c9ed-42b2-a9ed-c14f6cf14666 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '417c2e35727245fbbca6420395e50c44'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:06:03.516412 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-c2f9be5e-c9ed-42b2-a9ed-c14f6cf14666 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '417c2e35727245fbbca6420395e50c44'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:06:03.516960 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-c2f9be5e-c9ed-42b2-a9ed-c14f6cf14666 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '417c2e35727245fbbca6420395e50c44'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:06:03.530795 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-c2f9be5e-c9ed-42b2-a9ed-c14f6cf14666 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '417c2e35727245fbbca6420395e50c44'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:06:03.544705 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1038/2062] 10.222.0.40 () {58 vars in 1224 bytes} [Fri Aug 26 11:06:03 2022] DELETE /identity/v3/users/417c2e35727245fbbca6420395e50c44 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 11:06:03.549148 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad312bcc-dcbc-46d0-a04e-f2088bf29b2e None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:06:03.549623 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad312bcc-dcbc-46d0-a04e-f2088bf29b2e None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:06:03.549733 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad312bcc-dcbc-46d0-a04e-f2088bf29b2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:06:03.569510 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-ad312bcc-dcbc-46d0-a04e-f2088bf29b2e None None] MFA Rules not processed for user `ca155a48e2024c32af3fd2e19e5fac11`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 11:06:03.570736 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-ad312bcc-dcbc-46d0-a04e-f2088bf29b2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:06:03.633922 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1025/2063] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 11:06:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 11:06:03.644143 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f9be5e-c9ed-42b2-a9ed-c14f6cf14666 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:06:03.646506 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-c2f9be5e-c9ed-42b2-a9ed-c14f6cf14666 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:06:03.738501 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f9be5e-c9ed-42b2-a9ed-c14f6cf14666 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:06:03.744217 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123c4068-20ea-4ceb-b03d-b86b61baadbe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91da2acd2534423091537e1ba068d2b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e30ccf0c1204727b786876c141dbcd2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:06:03.744978 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123c4068-20ea-4ceb-b03d-b86b61baadbe service neutron] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:06:03.745264 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123c4068-20ea-4ceb-b03d-b86b61baadbe service neutron] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:06:03.745537 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123c4068-20ea-4ceb-b03d-b86b61baadbe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:06:03.763476 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-123c4068-20ea-4ceb-b03d-b86b61baadbe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:06:03.765212 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-123c4068-20ea-4ceb-b03d-b86b61baadbe service neutron] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:06:03.780924 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1039/2064] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 11:06:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 26 11:06:04.065369 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad312bcc-dcbc-46d0-a04e-f2088bf29b2e None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:06:04.066924 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-ad312bcc-dcbc-46d0-a04e-f2088bf29b2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:06:04.080158 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad312bcc-dcbc-46d0-a04e-f2088bf29b2e None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:06:04.085555 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aefa3ba5-5f3b-4eb4-a40c-8c87cecaed38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:06:04.086254 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefa3ba5-5f3b-4eb4-a40c-8c87cecaed38 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:06:04.086537 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefa3ba5-5f3b-4eb4-a40c-8c87cecaed38 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:06:04.086812 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefa3ba5-5f3b-4eb4-a40c-8c87cecaed38 admin admin] PATH_INFO: `/v3/projects/268541b52b674cb4a14e46236219cbdb` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:06:04.088012 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aefa3ba5-5f3b-4eb4-a40c-8c87cecaed38 admin admin] RBAC: Authorizing `identity:delete_project(project_id=268541b52b674cb4a14e46236219cbdb)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:06:04.090174 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aefa3ba5-5f3b-4eb4-a40c-8c87cecaed38 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:06:04.137171 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-aefa3ba5-5f3b-4eb4-a40c-8c87cecaed38 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '268541b52b674cb4a14e46236219cbdb', 'request_id': 'req-aefa3ba5-5f3b-4eb4-a40c-8c87cecaed38'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:06:04.143259 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-aefa3ba5-5f3b-4eb4-a40c-8c87cecaed38 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '268541b52b674cb4a14e46236219cbdb', 'request_id': 'req-aefa3ba5-5f3b-4eb4-a40c-8c87cecaed38'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:06:04.149482 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-aefa3ba5-5f3b-4eb4-a40c-8c87cecaed38 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '268541b52b674cb4a14e46236219cbdb', 'request_id': 'req-aefa3ba5-5f3b-4eb4-a40c-8c87cecaed38'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:06:04.162048 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-aefa3ba5-5f3b-4eb4-a40c-8c87cecaed38 admin admin] The token cache is being invalidate because project 268541b52b674cb4a14e46236219cbdb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81423) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 11:06:04.162370 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-aefa3ba5-5f3b-4eb4-a40c-8c87cecaed38 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:06:04.163023 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-aefa3ba5-5f3b-4eb4-a40c-8c87cecaed38 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:06:04.164537 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1026/2065] 10.222.0.40 () {58 vars in 1233 bytes} [Fri Aug 26 11:06:04 2022] DELETE /identity/v3/projects/268541b52b674cb4a14e46236219cbdb => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 11:06:04.169485 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cf6e3a-9a9f-4322-af96-d9333eda7ff1 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:06:04.169771 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cf6e3a-9a9f-4322-af96-d9333eda7ff1 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:06:04.170083 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cf6e3a-9a9f-4322-af96-d9333eda7ff1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:06:04.189176 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-28cf6e3a-9a9f-4322-af96-d9333eda7ff1 None None] MFA Rules not processed for user `ca155a48e2024c32af3fd2e19e5fac11`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 11:06:04.190413 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-28cf6e3a-9a9f-4322-af96-d9333eda7ff1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:06:04.200496 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 11:06:04.252315 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1040/2066] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 11:06:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 11:06:04.261069 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ac8910-7ee9-46a5-b38e-3ba6cf412244 None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:06:04.261423 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ac8910-7ee9-46a5-b38e-3ba6cf412244 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:06:04.261765 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ac8910-7ee9-46a5-b38e-3ba6cf412244 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:06:04.283740 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-61ac8910-7ee9-46a5-b38e-3ba6cf412244 None None] MFA Rules not processed for user `ca155a48e2024c32af3fd2e19e5fac11`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 11:06:04.285110 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-61ac8910-7ee9-46a5-b38e-3ba6cf412244 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:06:04.293901 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1027/2067] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 11:06:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 11:06:04.298322 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28cf6e3a-9a9f-4322-af96-d9333eda7ff1 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:06:04.308220 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28cf6e3a-9a9f-4322-af96-d9333eda7ff1 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:06:04.312900 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82bfcbf-6b94-4b5b-a724-f0428316ae3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:06:04.313494 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82bfcbf-6b94-4b5b-a724-f0428316ae3f admin admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:06:04.313857 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82bfcbf-6b94-4b5b-a724-f0428316ae3f admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:06:04.314133 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82bfcbf-6b94-4b5b-a724-f0428316ae3f admin admin] PATH_INFO: `/v3/domains` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:06:04.314722 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d82bfcbf-6b94-4b5b-a724-f0428316ae3f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 11:06:04.315019 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d82bfcbf-6b94-4b5b-a724-f0428316ae3f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:06:04.316601 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d82bfcbf-6b94-4b5b-a724-f0428316ae3f admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:06:04.330293 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1041/2068] 10.222.0.40 () {58 vars in 1153 bytes} [Fri Aug 26 11:06:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 11:06:04.336495 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ac8910-7ee9-46a5-b38e-3ba6cf412244 None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:06:04.346641 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ac8910-7ee9-46a5-b38e-3ba6cf412244 None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:06:04.351699 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef64318-b974-412f-9356-b7f771ef335e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:06:04.352301 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef64318-b974-412f-9356-b7f771ef335e admin admin] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:06:04.352573 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef64318-b974-412f-9356-b7f771ef335e admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:06:04.352837 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef64318-b974-412f-9356-b7f771ef335e admin admin] PATH_INFO: `/v3/projects` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:06:04.353457 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef64318-b974-412f-9356-b7f771ef335e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:06:04.355370 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef64318-b974-412f-9356-b7f771ef335e admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:06:04.383480 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1028/2069] 10.222.0.40 () {60 vars in 1153 bytes} [Fri Aug 26 11:06:04 2022] POST /identity/v3/projects => generated 375 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 11:06:04.387604 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82bfcbf-6b94-4b5b-a724-f0428316ae3f admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:06:04.399150 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82bfcbf-6b94-4b5b-a724-f0428316ae3f admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:06:04.455395 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ccf097-3524-4961-9cd5-5eaf96ed575e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:06:04.456000 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ccf097-3524-4961-9cd5-5eaf96ed575e admin admin] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:06:04.456282 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ccf097-3524-4961-9cd5-5eaf96ed575e admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:06:04.456558 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ccf097-3524-4961-9cd5-5eaf96ed575e admin admin] PATH_INFO: `/v3/users` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:06:04.457180 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ccf097-3524-4961-9cd5-5eaf96ed575e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:06:04.459137 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ccf097-3524-4961-9cd5-5eaf96ed575e admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:06:04.479465 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1042/2070] 10.222.0.40 () {60 vars in 1144 bytes} [Fri Aug 26 11:06:04 2022] POST /identity/v3/users => generated 288 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 11:06:04.483661 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef64318-b974-412f-9356-b7f771ef335e admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:06:04.495022 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef64318-b974-412f-9356-b7f771ef335e admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:06:04.499693 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3328883-27c2-4276-b292-02d46f2d0173 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:06:04.500284 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3328883-27c2-4276-b292-02d46f2d0173 admin admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:06:04.500561 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3328883-27c2-4276-b292-02d46f2d0173 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:06:04.500831 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3328883-27c2-4276-b292-02d46f2d0173 admin admin] PATH_INFO: `/v3/roles` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:06:04.501467 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3328883-27c2-4276-b292-02d46f2d0173 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:06:04.503086 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3328883-27c2-4276-b292-02d46f2d0173 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:06:04.513874 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1029/2071] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 26 11:06:04 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 11:06:04.517681 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ccf097-3524-4961-9cd5-5eaf96ed575e admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:06:04.527873 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ccf097-3524-4961-9cd5-5eaf96ed575e admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:06:04.532479 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a789334a-59b1-4a07-ad43-89c79d236831 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:06:04.533066 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a789334a-59b1-4a07-ad43-89c79d236831 admin admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:06:04.533349 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a789334a-59b1-4a07-ad43-89c79d236831 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:06:04.533622 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a789334a-59b1-4a07-ad43-89c79d236831 admin admin] PATH_INFO: `/v3/roles` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:06:04.534294 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a789334a-59b1-4a07-ad43-89c79d236831 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:06:04.535881 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a789334a-59b1-4a07-ad43-89c79d236831 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:06:04.547466 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1043/2072] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 26 11:06:04 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 11:06:04.551431 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3328883-27c2-4276-b292-02d46f2d0173 admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:06:04.562023 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3328883-27c2-4276-b292-02d46f2d0173 admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:06:04.567029 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d497e3a-af73-4e9e-9ad2-2129f9e23be1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:06:04.567668 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d497e3a-af73-4e9e-9ad2-2129f9e23be1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:06:04.568839 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d497e3a-af73-4e9e-9ad2-2129f9e23be1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:06:04.569224 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d497e3a-af73-4e9e-9ad2-2129f9e23be1 admin admin] PATH_INFO: `/v3/projects/d9e6c1c87a8347c888248e6c336942e5/users/dcc5e6c697b546dea8e2320a8c035f83/roles/23e174ce743b46eabe618c212691da55` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:06:04.584849 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d497e3a-af73-4e9e-9ad2-2129f9e23be1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d9e6c1c87a8347c888248e6c336942e5, user_id=dcc5e6c697b546dea8e2320a8c035f83, role_id=23e174ce743b46eabe618c212691da55)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:06:04.587575 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d497e3a-af73-4e9e-9ad2-2129f9e23be1 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:06:04.600764 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1030/2073] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 26 11:06:04 2022] PUT /identity/v3/projects/d9e6c1c87a8347c888248e6c336942e5/users/dcc5e6c697b546dea8e2320a8c035f83/roles/23e174ce743b46eabe618c212691da55 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 26 11:06:04.612221 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df24b2f0-89cb-4780-85e2-0b84f84bac54 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:06:04.612505 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df24b2f0-89cb-4780-85e2-0b84f84bac54 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:06:04.612779 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df24b2f0-89cb-4780-85e2-0b84f84bac54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:06:04.640964 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-df24b2f0-89cb-4780-85e2-0b84f84bac54 None None] MFA Rules not processed for user `dcc5e6c697b546dea8e2320a8c035f83`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 11:06:04.642213 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-df24b2f0-89cb-4780-85e2-0b84f84bac54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:06:04.704502 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1044/2074] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 11:06:04 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 11:06:04.715172 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d497e3a-af73-4e9e-9ad2-2129f9e23be1 admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:06:04.716888 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-0d497e3a-af73-4e9e-9ad2-2129f9e23be1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:06:04.727919 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 11:06:04.776287 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d497e3a-af73-4e9e-9ad2-2129f9e23be1 admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:06:04.779888 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59978e25-d962-4cb6-bb75-f3a3d309f0d5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c277338a7f814cc6b01c5419775031bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e30ccf0c1204727b786876c141dbcd2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:06:04.780487 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59978e25-d962-4cb6-bb75-f3a3d309f0d5 service cinder] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:06:04.780798 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59978e25-d962-4cb6-bb75-f3a3d309f0d5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:06:04.781075 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59978e25-d962-4cb6-bb75-f3a3d309f0d5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:06:04.790001 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59978e25-d962-4cb6-bb75-f3a3d309f0d5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:06:04.791725 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59978e25-d962-4cb6-bb75-f3a3d309f0d5 service cinder] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:06:04.801589 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1031/2075] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 11:06:04 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 11:06:07.195850 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df24b2f0-89cb-4780-85e2-0b84f84bac54 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:06:07.197315 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-df24b2f0-89cb-4780-85e2-0b84f84bac54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:06:07.212911 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 11:06:07.314192 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df24b2f0-89cb-4780-85e2-0b84f84bac54 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:06:07.326873 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23009460-c5da-4a02-8214-81620758a794 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:06:07.328998 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23009460-c5da-4a02-8214-81620758a794 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:06:07.328998 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23009460-c5da-4a02-8214-81620758a794 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:06:07.328998 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23009460-c5da-4a02-8214-81620758a794 admin admin] PATH_INFO: `/v3/users/2e03effab5f54b648287cb93cff0a8a0` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:06:07.333756 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23009460-c5da-4a02-8214-81620758a794 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2e03effab5f54b648287cb93cff0a8a0)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:06:07.333756 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23009460-c5da-4a02-8214-81620758a794 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:06:07.427146 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-23009460-c5da-4a02-8214-81620758a794 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2e03effab5f54b648287cb93cff0a8a0'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:06:07.429396 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-23009460-c5da-4a02-8214-81620758a794 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2e03effab5f54b648287cb93cff0a8a0'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:06:07.443633 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-23009460-c5da-4a02-8214-81620758a794 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2e03effab5f54b648287cb93cff0a8a0'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:06:07.444643 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-23009460-c5da-4a02-8214-81620758a794 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2e03effab5f54b648287cb93cff0a8a0'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:06:07.463984 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-23009460-c5da-4a02-8214-81620758a794 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2e03effab5f54b648287cb93cff0a8a0'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:06:07.492648 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1045/2076] 10.222.0.40 () {58 vars in 1224 bytes} [Fri Aug 26 11:06:07 2022] DELETE /identity/v3/users/2e03effab5f54b648287cb93cff0a8a0 => generated 0 bytes in 297 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 11:06:07.497192 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef68e959-7c39-45fa-bf15-ccd95a1b5067 None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:06:07.497688 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef68e959-7c39-45fa-bf15-ccd95a1b5067 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:06:07.497774 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef68e959-7c39-45fa-bf15-ccd95a1b5067 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:06:07.524502 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-ef68e959-7c39-45fa-bf15-ccd95a1b5067 None None] MFA Rules not processed for user `ca155a48e2024c32af3fd2e19e5fac11`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 11:06:07.526431 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-ef68e959-7c39-45fa-bf15-ccd95a1b5067 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:06:07.600325 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1032/2077] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 11:06:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 11:06:07.612561 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23009460-c5da-4a02-8214-81620758a794 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:06:07.615930 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-23009460-c5da-4a02-8214-81620758a794 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:06:07.707952 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23009460-c5da-4a02-8214-81620758a794 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:06:07.715907 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b78fb3d-cf11-4fbf-aa7c-1e68116d219b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91da2acd2534423091537e1ba068d2b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e30ccf0c1204727b786876c141dbcd2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:06:07.716645 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b78fb3d-cf11-4fbf-aa7c-1e68116d219b service neutron] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:06:07.718057 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b78fb3d-cf11-4fbf-aa7c-1e68116d219b service neutron] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:06:07.718445 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b78fb3d-cf11-4fbf-aa7c-1e68116d219b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:06:07.734814 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b78fb3d-cf11-4fbf-aa7c-1e68116d219b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:06:07.736406 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b78fb3d-cf11-4fbf-aa7c-1e68116d219b service neutron] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:06:07.758553 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1046/2078] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 11:06:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 11:06:08.111575 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef68e959-7c39-45fa-bf15-ccd95a1b5067 None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:06:08.113473 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-ef68e959-7c39-45fa-bf15-ccd95a1b5067 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:06:08.135238 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef68e959-7c39-45fa-bf15-ccd95a1b5067 None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:06:08.141695 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839d4dcf-f106-4ab5-b40f-bca2ac970afb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:06:08.142614 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839d4dcf-f106-4ab5-b40f-bca2ac970afb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:06:08.143066 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839d4dcf-f106-4ab5-b40f-bca2ac970afb admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:06:08.143513 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839d4dcf-f106-4ab5-b40f-bca2ac970afb admin admin] PATH_INFO: `/v3/projects/ecdd6d54de5e47df9a0ecaa4c836b7fc` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:06:08.144924 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-839d4dcf-f106-4ab5-b40f-bca2ac970afb admin admin] RBAC: Authorizing `identity:delete_project(project_id=ecdd6d54de5e47df9a0ecaa4c836b7fc)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:06:08.147509 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-839d4dcf-f106-4ab5-b40f-bca2ac970afb admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:06:08.211107 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-839d4dcf-f106-4ab5-b40f-bca2ac970afb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ecdd6d54de5e47df9a0ecaa4c836b7fc', 'request_id': 'req-839d4dcf-f106-4ab5-b40f-bca2ac970afb'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:06:08.219154 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-839d4dcf-f106-4ab5-b40f-bca2ac970afb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ecdd6d54de5e47df9a0ecaa4c836b7fc', 'request_id': 'req-839d4dcf-f106-4ab5-b40f-bca2ac970afb'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:06:08.227481 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-839d4dcf-f106-4ab5-b40f-bca2ac970afb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ecdd6d54de5e47df9a0ecaa4c836b7fc', 'request_id': 'req-839d4dcf-f106-4ab5-b40f-bca2ac970afb'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:06:08.243928 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-839d4dcf-f106-4ab5-b40f-bca2ac970afb admin admin] The token cache is being invalidate because project ecdd6d54de5e47df9a0ecaa4c836b7fc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81423) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 11:06:08.244741 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-839d4dcf-f106-4ab5-b40f-bca2ac970afb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:06:08.245400 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-839d4dcf-f106-4ab5-b40f-bca2ac970afb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:06:08.246944 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1033/2079] 10.222.0.40 () {58 vars in 1233 bytes} [Fri Aug 26 11:06:08 2022] DELETE /identity/v3/projects/ecdd6d54de5e47df9a0ecaa4c836b7fc => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 11:06:08.251105 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b78fb3d-cf11-4fbf-aa7c-1e68116d219b service neutron] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:06:08.263213 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b78fb3d-cf11-4fbf-aa7c-1e68116d219b service neutron] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:06:08.299627 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 11:06:08.331473 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d83ffa8-eb9e-48a2-97dc-2a06ad5d0bc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:06:08.332106 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d83ffa8-eb9e-48a2-97dc-2a06ad5d0bc8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:06:08.332385 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d83ffa8-eb9e-48a2-97dc-2a06ad5d0bc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:06:08.332653 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d83ffa8-eb9e-48a2-97dc-2a06ad5d0bc8 admin admin] PATH_INFO: `/v3/users/7e8232f1071241158ac6101404b9bdc9` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:06:08.334540 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d83ffa8-eb9e-48a2-97dc-2a06ad5d0bc8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7e8232f1071241158ac6101404b9bdc9)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:06:08.337327 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d83ffa8-eb9e-48a2-97dc-2a06ad5d0bc8 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:06:08.393688 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-3d83ffa8-eb9e-48a2-97dc-2a06ad5d0bc8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7e8232f1071241158ac6101404b9bdc9'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:06:08.394376 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-3d83ffa8-eb9e-48a2-97dc-2a06ad5d0bc8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7e8232f1071241158ac6101404b9bdc9'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:06:08.401934 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-3d83ffa8-eb9e-48a2-97dc-2a06ad5d0bc8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7e8232f1071241158ac6101404b9bdc9'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:06:08.402593 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-3d83ffa8-eb9e-48a2-97dc-2a06ad5d0bc8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7e8232f1071241158ac6101404b9bdc9'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:06:08.415607 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-3d83ffa8-eb9e-48a2-97dc-2a06ad5d0bc8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7e8232f1071241158ac6101404b9bdc9'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:06:08.429019 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1047/2080] 10.222.0.40 () {58 vars in 1224 bytes} [Fri Aug 26 11:06:08 2022] DELETE /identity/v3/users/7e8232f1071241158ac6101404b9bdc9 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 11:06:08.784234 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839d4dcf-f106-4ab5-b40f-bca2ac970afb admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:06:08.785854 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-839d4dcf-f106-4ab5-b40f-bca2ac970afb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:06:08.804542 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839d4dcf-f106-4ab5-b40f-bca2ac970afb admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:06:08.840799 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 11:06:08.869831 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4dc82c-2947-418b-9bec-bb4aeec90af8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:06:08.870884 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4dc82c-2947-418b-9bec-bb4aeec90af8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:06:08.871319 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4dc82c-2947-418b-9bec-bb4aeec90af8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:06:08.871755 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4dc82c-2947-418b-9bec-bb4aeec90af8 admin admin] PATH_INFO: `/v3/projects/245cdfba492848b6a2350a9e0342cce0` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:06:08.873052 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc4dc82c-2947-418b-9bec-bb4aeec90af8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=245cdfba492848b6a2350a9e0342cce0)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:06:08.875828 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc4dc82c-2947-418b-9bec-bb4aeec90af8 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:06:08.929052 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-dc4dc82c-2947-418b-9bec-bb4aeec90af8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '245cdfba492848b6a2350a9e0342cce0', 'request_id': 'req-dc4dc82c-2947-418b-9bec-bb4aeec90af8'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:06:08.936146 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-dc4dc82c-2947-418b-9bec-bb4aeec90af8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '245cdfba492848b6a2350a9e0342cce0', 'request_id': 'req-dc4dc82c-2947-418b-9bec-bb4aeec90af8'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:06:08.943638 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-dc4dc82c-2947-418b-9bec-bb4aeec90af8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '245cdfba492848b6a2350a9e0342cce0', 'request_id': 'req-dc4dc82c-2947-418b-9bec-bb4aeec90af8'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:06:08.957770 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-dc4dc82c-2947-418b-9bec-bb4aeec90af8 admin admin] The token cache is being invalidate because project 245cdfba492848b6a2350a9e0342cce0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81423) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 11:06:08.958147 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-dc4dc82c-2947-418b-9bec-bb4aeec90af8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:06:08.959348 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-dc4dc82c-2947-418b-9bec-bb4aeec90af8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:06:08.961328 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1034/2081] 10.222.0.40 () {58 vars in 1233 bytes} [Fri Aug 26 11:06:08 2022] DELETE /identity/v3/projects/245cdfba492848b6a2350a9e0342cce0 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 11:06:08.966612 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ebcc4a-ca5e-4ec8-8135-00f3a5a24cf3 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:06:08.966954 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ebcc4a-ca5e-4ec8-8135-00f3a5a24cf3 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:06:08.967239 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ebcc4a-ca5e-4ec8-8135-00f3a5a24cf3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:06:08.989002 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-93ebcc4a-ca5e-4ec8-8135-00f3a5a24cf3 None None] MFA Rules not processed for user `ca155a48e2024c32af3fd2e19e5fac11`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 11:06:08.990927 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-93ebcc4a-ca5e-4ec8-8135-00f3a5a24cf3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:06:09.061750 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1048/2082] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 11:06:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 11:06:09.069656 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0583d4b7-25a6-48ab-85b6-d0c5dd992f60 None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:06:09.070073 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0583d4b7-25a6-48ab-85b6-d0c5dd992f60 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:06:09.070456 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0583d4b7-25a6-48ab-85b6-d0c5dd992f60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:06:09.091596 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-0583d4b7-25a6-48ab-85b6-d0c5dd992f60 None None] MFA Rules not processed for user `ca155a48e2024c32af3fd2e19e5fac11`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 11:06:09.092645 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-0583d4b7-25a6-48ab-85b6-d0c5dd992f60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:06:09.102127 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1035/2083] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 11:06:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 11:06:09.108523 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ebcc4a-ca5e-4ec8-8135-00f3a5a24cf3 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:06:09.117586 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ebcc4a-ca5e-4ec8-8135-00f3a5a24cf3 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:06:09.122775 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728404f6-0eb0-4156-90fa-7e13fb080c10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:06:09.123383 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728404f6-0eb0-4156-90fa-7e13fb080c10 admin admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:06:09.123668 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728404f6-0eb0-4156-90fa-7e13fb080c10 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:06:09.123941 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728404f6-0eb0-4156-90fa-7e13fb080c10 admin admin] PATH_INFO: `/v3/domains` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:06:09.124589 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-728404f6-0eb0-4156-90fa-7e13fb080c10 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 11:06:09.124891 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-728404f6-0eb0-4156-90fa-7e13fb080c10 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:06:09.126536 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-728404f6-0eb0-4156-90fa-7e13fb080c10 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:06:09.139512 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1049/2084] 10.222.0.40 () {58 vars in 1153 bytes} [Fri Aug 26 11:06:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 11:06:09.145237 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0583d4b7-25a6-48ab-85b6-d0c5dd992f60 None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:06:09.157447 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0583d4b7-25a6-48ab-85b6-d0c5dd992f60 None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:06:09.163191 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e543a876-ad71-4485-bccf-11bc3723f573 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:06:09.163799 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e543a876-ad71-4485-bccf-11bc3723f573 admin admin] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:06:09.164174 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e543a876-ad71-4485-bccf-11bc3723f573 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:06:09.164541 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e543a876-ad71-4485-bccf-11bc3723f573 admin admin] PATH_INFO: `/v3/projects` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:06:09.165374 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e543a876-ad71-4485-bccf-11bc3723f573 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:06:09.167396 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e543a876-ad71-4485-bccf-11bc3723f573 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:06:09.207275 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1036/2085] 10.222.0.40 () {60 vars in 1153 bytes} [Fri Aug 26 11:06:09 2022] POST /identity/v3/projects => generated 369 bytes in 64 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 11:06:09.211628 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728404f6-0eb0-4156-90fa-7e13fb080c10 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:06:09.222624 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728404f6-0eb0-4156-90fa-7e13fb080c10 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:06:09.265169 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 11:06:09.301799 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e543a876-ad71-4485-bccf-11bc3723f573 admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:06:09.304824 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-e543a876-ad71-4485-bccf-11bc3723f573 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:06:09.339554 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ce24d9-3855-47ce-9908-7de4ea239fdb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:06:09.340983 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ce24d9-3855-47ce-9908-7de4ea239fdb admin admin] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:06:09.341864 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ce24d9-3855-47ce-9908-7de4ea239fdb admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:06:09.344142 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ce24d9-3855-47ce-9908-7de4ea239fdb admin admin] PATH_INFO: `/v3/users` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:06:09.345029 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ce24d9-3855-47ce-9908-7de4ea239fdb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:06:09.351314 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ce24d9-3855-47ce-9908-7de4ea239fdb admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:06:09.405076 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1050/2086] 10.222.0.40 () {60 vars in 1144 bytes} [Fri Aug 26 11:06:09 2022] POST /identity/v3/users => generated 285 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 11:06:09.407193 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ce24d9-3855-47ce-9908-7de4ea239fdb admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:06:09.425840 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ce24d9-3855-47ce-9908-7de4ea239fdb admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:06:09.432653 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35d099d-1169-45f3-bba7-14f7b8718ed5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:06:09.433699 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35d099d-1169-45f3-bba7-14f7b8718ed5 admin admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:06:09.434167 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35d099d-1169-45f3-bba7-14f7b8718ed5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:06:09.434574 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35d099d-1169-45f3-bba7-14f7b8718ed5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:06:09.435591 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a35d099d-1169-45f3-bba7-14f7b8718ed5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:06:09.442473 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a35d099d-1169-45f3-bba7-14f7b8718ed5 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:06:09.452340 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e543a876-ad71-4485-bccf-11bc3723f573 admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:06:09.458061 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1051/2087] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 26 11:06:09 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 26 11:06:09.463156 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24660ec3-3510-409f-b4d6-3491dc3e9f51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:06:09.463336 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35d099d-1169-45f3-bba7-14f7b8718ed5 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:06:09.463872 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24660ec3-3510-409f-b4d6-3491dc3e9f51 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:06:09.464152 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24660ec3-3510-409f-b4d6-3491dc3e9f51 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:06:09.464421 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24660ec3-3510-409f-b4d6-3491dc3e9f51 admin admin] PATH_INFO: `/v3/users/dcc5e6c697b546dea8e2320a8c035f83` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:06:09.467602 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24660ec3-3510-409f-b4d6-3491dc3e9f51 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dcc5e6c697b546dea8e2320a8c035f83)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:06:09.469737 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24660ec3-3510-409f-b4d6-3491dc3e9f51 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:06:09.476924 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35d099d-1169-45f3-bba7-14f7b8718ed5 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:06:09.491327 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b9ecd4-5c0e-4e65-87ea-74edc4796dff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:06:09.491944 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b9ecd4-5c0e-4e65-87ea-74edc4796dff admin admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:06:09.492221 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b9ecd4-5c0e-4e65-87ea-74edc4796dff admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:06:09.492746 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b9ecd4-5c0e-4e65-87ea-74edc4796dff admin admin] PATH_INFO: `/v3/roles` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:06:09.493393 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b9ecd4-5c0e-4e65-87ea-74edc4796dff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:06:09.495009 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b9ecd4-5c0e-4e65-87ea-74edc4796dff admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:06:09.506985 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1052/2088] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 26 11:06:09 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 11:06:09.511388 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b9ecd4-5c0e-4e65-87ea-74edc4796dff admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:06:09.524115 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b9ecd4-5c0e-4e65-87ea-74edc4796dff admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:06:09.529044 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de075ab-1aed-4569-badb-d7e631361a6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:06:09.529776 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de075ab-1aed-4569-badb-d7e631361a6b admin admin] REQUEST_METHOD: `PUT` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:06:09.530159 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de075ab-1aed-4569-badb-d7e631361a6b admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:06:09.530514 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de075ab-1aed-4569-badb-d7e631361a6b admin admin] PATH_INFO: `/v3/projects/178703c6da2441a2bf91d5d855302d9c/users/28be7d3acfca4556b9b2769428815404/roles/23e174ce743b46eabe618c212691da55` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:06:09.532689 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 11:06:09.542969 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-24660ec3-3510-409f-b4d6-3491dc3e9f51 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dcc5e6c697b546dea8e2320a8c035f83'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:06:09.551348 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de075ab-1aed-4569-badb-d7e631361a6b admin admin] RBAC: Authorizing `identity:create_grant(project_id=178703c6da2441a2bf91d5d855302d9c, user_id=28be7d3acfca4556b9b2769428815404, role_id=23e174ce743b46eabe618c212691da55)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:06:09.552286 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-24660ec3-3510-409f-b4d6-3491dc3e9f51 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dcc5e6c697b546dea8e2320a8c035f83'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:06:09.553961 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-24660ec3-3510-409f-b4d6-3491dc3e9f51 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dcc5e6c697b546dea8e2320a8c035f83'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:06:09.554638 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de075ab-1aed-4569-badb-d7e631361a6b admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:06:09.568374 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1053/2089] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 26 11:06:09 2022] PUT /identity/v3/projects/178703c6da2441a2bf91d5d855302d9c/users/28be7d3acfca4556b9b2769428815404/roles/23e174ce743b46eabe618c212691da55 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 11:06:09.571532 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-24660ec3-3510-409f-b4d6-3491dc3e9f51 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dcc5e6c697b546dea8e2320a8c035f83'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:06:09.572489 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-24660ec3-3510-409f-b4d6-3491dc3e9f51 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dcc5e6c697b546dea8e2320a8c035f83'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:06:09.579310 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e9d178-0aef-474e-9bfa-e3af14eb034c None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:06:09.579590 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e9d178-0aef-474e-9bfa-e3af14eb034c None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:06:09.579859 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e9d178-0aef-474e-9bfa-e3af14eb034c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:06:09.593495 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1037/2090] 10.222.0.40 () {58 vars in 1224 bytes} [Fri Aug 26 11:06:09 2022] DELETE /identity/v3/users/dcc5e6c697b546dea8e2320a8c035f83 => generated 0 bytes in 292 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 11:06:09.598823 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2068f2fb-f2c4-4b6c-837e-9c6369220f85 None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:06:09.599794 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2068f2fb-f2c4-4b6c-837e-9c6369220f85 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:06:09.600332 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2068f2fb-f2c4-4b6c-837e-9c6369220f85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:06:09.612354 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-56e9d178-0aef-474e-9bfa-e3af14eb034c None None] MFA Rules not processed for user `28be7d3acfca4556b9b2769428815404`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 11:06:09.613574 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-56e9d178-0aef-474e-9bfa-e3af14eb034c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:06:09.621768 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-2068f2fb-f2c4-4b6c-837e-9c6369220f85 None None] MFA Rules not processed for user `ca155a48e2024c32af3fd2e19e5fac11`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 11:06:09.623277 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-2068f2fb-f2c4-4b6c-837e-9c6369220f85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:06:09.694127 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1054/2091] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 11:06:09 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 11:06:09.698932 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e9d178-0aef-474e-9bfa-e3af14eb034c None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:06:09.700328 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-56e9d178-0aef-474e-9bfa-e3af14eb034c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:06:09.704110 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1038/2092] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 11:06:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 11:06:09.711969 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2068f2fb-f2c4-4b6c-837e-9c6369220f85 None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:06:09.713628 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-2068f2fb-f2c4-4b6c-837e-9c6369220f85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:06:09.715549 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e9d178-0aef-474e-9bfa-e3af14eb034c None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:06:09.721782 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513674cb-040e-4604-bb41-1a8db2d03011 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:06:09.722763 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513674cb-040e-4604-bb41-1a8db2d03011 admin admin] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:06:09.723043 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513674cb-040e-4604-bb41-1a8db2d03011 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:06:09.723692 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513674cb-040e-4604-bb41-1a8db2d03011 admin admin] PATH_INFO: `/v3/projects` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:06:09.725851 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-513674cb-040e-4604-bb41-1a8db2d03011 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:06:09.727949 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-513674cb-040e-4604-bb41-1a8db2d03011 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:06:09.763235 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1055/2093] 10.222.0.40 () {60 vars in 1153 bytes} [Fri Aug 26 11:06:09 2022] POST /identity/v3/projects => generated 367 bytes in 66 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 26 11:06:09.767009 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513674cb-040e-4604-bb41-1a8db2d03011 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:06:09.802490 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2068f2fb-f2c4-4b6c-837e-9c6369220f85 None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:06:09.841276 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 11:06:09.862012 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513674cb-040e-4604-bb41-1a8db2d03011 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:06:09.871652 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd75027-8375-4c39-a2df-9e86feaa7658 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:06:09.872536 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd75027-8375-4c39-a2df-9e86feaa7658 admin admin] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:06:09.873217 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd75027-8375-4c39-a2df-9e86feaa7658 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:06:09.873584 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd75027-8375-4c39-a2df-9e86feaa7658 admin admin] PATH_INFO: `/v3/users` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:06:09.874363 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd75027-8375-4c39-a2df-9e86feaa7658 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:06:09.876773 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd75027-8375-4c39-a2df-9e86feaa7658 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:06:09.878923 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d3617c-c47d-485d-b16a-b0f586fd2050 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91da2acd2534423091537e1ba068d2b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e30ccf0c1204727b786876c141dbcd2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:06:09.879792 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d3617c-c47d-485d-b16a-b0f586fd2050 service neutron] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:06:09.880199 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d3617c-c47d-485d-b16a-b0f586fd2050 service neutron] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:06:09.880497 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d3617c-c47d-485d-b16a-b0f586fd2050 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:06:09.896019 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d3617c-c47d-485d-b16a-b0f586fd2050 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:06:09.898451 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d3617c-c47d-485d-b16a-b0f586fd2050 service neutron] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:06:09.903045 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1056/2094] 10.222.0.40 () {60 vars in 1144 bytes} [Fri Aug 26 11:06:09 2022] POST /identity/v3/users => generated 290 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 11:06:09.906834 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd75027-8375-4c39-a2df-9e86feaa7658 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:06:09.912714 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1039/2095] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 11:06:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 202 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 26 11:06:09.917857 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd75027-8375-4c39-a2df-9e86feaa7658 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:06:09.922845 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a995445-8266-4667-a649-fadaad47e7b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:06:09.923458 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a995445-8266-4667-a649-fadaad47e7b0 admin admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:06:09.923743 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a995445-8266-4667-a649-fadaad47e7b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:06:09.924017 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a995445-8266-4667-a649-fadaad47e7b0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:06:09.924663 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a995445-8266-4667-a649-fadaad47e7b0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:06:09.926487 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a995445-8266-4667-a649-fadaad47e7b0 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:06:09.938656 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1057/2096] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 26 11:06:09 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 26 11:06:09.942615 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d3617c-c47d-485d-b16a-b0f586fd2050 service neutron] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:06:09.959595 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d3617c-c47d-485d-b16a-b0f586fd2050 service neutron] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:06:09.969556 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4487a6bd-d815-45c7-9142-2ad53041cad2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:06:09.970286 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4487a6bd-d815-45c7-9142-2ad53041cad2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:06:09.972311 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4487a6bd-d815-45c7-9142-2ad53041cad2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:06:09.972399 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4487a6bd-d815-45c7-9142-2ad53041cad2 admin admin] PATH_INFO: `/v3/projects/e8a9b964e5f44287a0c3825eeb06e042/users/75052b4fe894452e97d03444bf26641e/roles/1f97e4602e2e4d6498913a81f9ae8d6a` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:06:09.993549 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:DELETE=5,keystone:SELECT=94,keystone:INSERT=1 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 11:06:09.999969 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4487a6bd-d815-45c7-9142-2ad53041cad2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e8a9b964e5f44287a0c3825eeb06e042, user_id=75052b4fe894452e97d03444bf26641e, role_id=1f97e4602e2e4d6498913a81f9ae8d6a)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:06:10.006181 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4487a6bd-d815-45c7-9142-2ad53041cad2 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:06:10.036421 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1040/2097] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 26 11:06:09 2022] PUT /identity/v3/projects/e8a9b964e5f44287a0c3825eeb06e042/users/75052b4fe894452e97d03444bf26641e/roles/1f97e4602e2e4d6498913a81f9ae8d6a => generated 0 bytes in 94 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 11:06:10.040814 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a995445-8266-4667-a649-fadaad47e7b0 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:06:10.058603 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a995445-8266-4667-a649-fadaad47e7b0 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:06:10.152914 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0ac9e1-c5fd-4059-8f02-580a0b78011b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:06:10.153718 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0ac9e1-c5fd-4059-8f02-580a0b78011b admin admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:06:10.153998 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0ac9e1-c5fd-4059-8f02-580a0b78011b admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:06:10.154335 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0ac9e1-c5fd-4059-8f02-580a0b78011b admin admin] PATH_INFO: `/v3/roles` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:06:10.155136 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa0ac9e1-c5fd-4059-8f02-580a0b78011b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:06:10.156756 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa0ac9e1-c5fd-4059-8f02-580a0b78011b admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:06:10.180982 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1058/2098] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 26 11:06:10 2022] GET /identity/v3/roles => generated 1420 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 11:06:10.182245 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4487a6bd-d815-45c7-9142-2ad53041cad2 admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:06:10.194281 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4487a6bd-d815-45c7-9142-2ad53041cad2 admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:06:10.199494 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d837bae4-fe21-44be-a260-6a91af02ed95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:06:10.200180 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d837bae4-fe21-44be-a260-6a91af02ed95 admin admin] REQUEST_METHOD: `PUT` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:06:10.200462 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d837bae4-fe21-44be-a260-6a91af02ed95 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:06:10.200737 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d837bae4-fe21-44be-a260-6a91af02ed95 admin admin] PATH_INFO: `/v3/projects/e8a9b964e5f44287a0c3825eeb06e042/users/75052b4fe894452e97d03444bf26641e/roles/1f97e4602e2e4d6498913a81f9ae8d6a` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:06:10.202847 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d837bae4-fe21-44be-a260-6a91af02ed95 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e8a9b964e5f44287a0c3825eeb06e042, user_id=75052b4fe894452e97d03444bf26641e, role_id=1f97e4602e2e4d6498913a81f9ae8d6a)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:06:10.205934 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d837bae4-fe21-44be-a260-6a91af02ed95 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:06:10.228976 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1041/2099] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 26 11:06:10 2022] PUT /identity/v3/projects/e8a9b964e5f44287a0c3825eeb06e042/users/75052b4fe894452e97d03444bf26641e/roles/1f97e4602e2e4d6498913a81f9ae8d6a => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 11:06:10.240654 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e4eef7-4689-4403-aeec-2b2e802518bd None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:06:10.240916 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e4eef7-4689-4403-aeec-2b2e802518bd None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:06:10.241174 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e4eef7-4689-4403-aeec-2b2e802518bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:06:10.267554 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d837bae4-fe21-44be-a260-6a91af02ed95 admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:06:10.269261 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-d837bae4-fe21-44be-a260-6a91af02ed95 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:06:10.283326 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-92e4eef7-4689-4403-aeec-2b2e802518bd None None] MFA Rules not processed for user `75052b4fe894452e97d03444bf26641e`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 11:06:10.285372 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-92e4eef7-4689-4403-aeec-2b2e802518bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:06:10.374898 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d837bae4-fe21-44be-a260-6a91af02ed95 admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:06:10.380443 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960a35bf-01ac-45a9-bf73-fb5d574241ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:06:10.381490 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960a35bf-01ac-45a9-bf73-fb5d574241ad admin admin] REQUEST_METHOD: `DELETE` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:06:10.381949 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960a35bf-01ac-45a9-bf73-fb5d574241ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:06:10.383022 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960a35bf-01ac-45a9-bf73-fb5d574241ad admin admin] PATH_INFO: `/v3/projects/d9e6c1c87a8347c888248e6c336942e5` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:06:10.384489 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-960a35bf-01ac-45a9-bf73-fb5d574241ad admin admin] RBAC: Authorizing `identity:delete_project(project_id=d9e6c1c87a8347c888248e6c336942e5)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:06:10.387301 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-960a35bf-01ac-45a9-bf73-fb5d574241ad admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:06:10.398328 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1059/2100] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 11:06:10 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 11:06:10.405755 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e4eef7-4689-4403-aeec-2b2e802518bd None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:06:10.407610 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-92e4eef7-4689-4403-aeec-2b2e802518bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:06:10.467128 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-960a35bf-01ac-45a9-bf73-fb5d574241ad admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd9e6c1c87a8347c888248e6c336942e5', 'request_id': 'req-960a35bf-01ac-45a9-bf73-fb5d574241ad'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:06:10.475135 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-960a35bf-01ac-45a9-bf73-fb5d574241ad admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd9e6c1c87a8347c888248e6c336942e5', 'request_id': 'req-960a35bf-01ac-45a9-bf73-fb5d574241ad'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:06:10.489642 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e4eef7-4689-4403-aeec-2b2e802518bd None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:06:10.494973 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 11:06:10.502435 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-960a35bf-01ac-45a9-bf73-fb5d574241ad admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd9e6c1c87a8347c888248e6c336942e5', 'request_id': 'req-960a35bf-01ac-45a9-bf73-fb5d574241ad'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:06:10.523518 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-960a35bf-01ac-45a9-bf73-fb5d574241ad admin admin] The token cache is being invalidate because project d9e6c1c87a8347c888248e6c336942e5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81423) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 11:06:10.523856 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-960a35bf-01ac-45a9-bf73-fb5d574241ad admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:06:10.524654 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-960a35bf-01ac-45a9-bf73-fb5d574241ad admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:06:10.527095 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1042/2101] 10.222.0.40 () {58 vars in 1233 bytes} [Fri Aug 26 11:06:10 2022] DELETE /identity/v3/projects/d9e6c1c87a8347c888248e6c336942e5 => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 26 11:06:10.532811 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b18424-d429-4c96-a546-a86e9939c4a7 None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:06:10.533095 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b18424-d429-4c96-a546-a86e9939c4a7 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:06:10.533365 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b18424-d429-4c96-a546-a86e9939c4a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:06:10.562456 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-98b18424-d429-4c96-a546-a86e9939c4a7 None None] MFA Rules not processed for user `ca155a48e2024c32af3fd2e19e5fac11`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 11:06:10.564550 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-98b18424-d429-4c96-a546-a86e9939c4a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:06:10.569900 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a34e091-6500-4745-a3a3-3b5ada3e183f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c277338a7f814cc6b01c5419775031bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e30ccf0c1204727b786876c141dbcd2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:06:10.570653 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a34e091-6500-4745-a3a3-3b5ada3e183f service cinder] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:06:10.571045 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a34e091-6500-4745-a3a3-3b5ada3e183f service cinder] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:06:10.571425 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a34e091-6500-4745-a3a3-3b5ada3e183f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:06:10.577961 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-6a34e091-6500-4745-a3a3-3b5ada3e183f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:06:10.657930 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1043/2102] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 11:06:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 11:06:10.667756 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861a2a48-8b57-4a84-a193-579f1671d4e9 None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:06:10.668053 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861a2a48-8b57-4a84-a193-579f1671d4e9 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:06:10.668327 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861a2a48-8b57-4a84-a193-579f1671d4e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:06:10.671055 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a34e091-6500-4745-a3a3-3b5ada3e183f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:06:10.673517 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a34e091-6500-4745-a3a3-3b5ada3e183f service cinder] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:06:10.688972 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1060/2103] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 11:06:10 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 284 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 11:06:10.692697 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-861a2a48-8b57-4a84-a193-579f1671d4e9 None None] MFA Rules not processed for user `ca155a48e2024c32af3fd2e19e5fac11`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 11:06:10.694695 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-861a2a48-8b57-4a84-a193-579f1671d4e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:06:10.708778 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1044/2104] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 11:06:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 11:06:10.722167 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a34e091-6500-4745-a3a3-3b5ada3e183f service cinder] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:06:10.736859 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a34e091-6500-4745-a3a3-3b5ada3e183f service cinder] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:06:10.754188 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf14219-043c-4b26-9110-f8a39da96f60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:06:10.754946 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf14219-043c-4b26-9110-f8a39da96f60 admin admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:06:10.755359 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf14219-043c-4b26-9110-f8a39da96f60 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:06:10.756527 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf14219-043c-4b26-9110-f8a39da96f60 admin admin] PATH_INFO: `/v3/domains` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:06:10.756773 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861a2a48-8b57-4a84-a193-579f1671d4e9 None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:06:10.759437 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-861a2a48-8b57-4a84-a193-579f1671d4e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:06:10.759898 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf14219-043c-4b26-9110-f8a39da96f60 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 11:06:10.760317 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf14219-043c-4b26-9110-f8a39da96f60 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:06:10.762098 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf14219-043c-4b26-9110-f8a39da96f60 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:06:10.769034 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 11:06:10.784081 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861a2a48-8b57-4a84-a193-579f1671d4e9 None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:06:10.790619 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1061/2105] 10.222.0.40 () {58 vars in 1153 bytes} [Fri Aug 26 11:06:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 11:06:10.795187 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf14219-043c-4b26-9110-f8a39da96f60 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:06:10.796075 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7edf2d9b-0d40-455e-855c-ad5c751a51ff service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c277338a7f814cc6b01c5419775031bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e30ccf0c1204727b786876c141dbcd2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:06:10.797774 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7edf2d9b-0d40-455e-855c-ad5c751a51ff service cinder] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:06:10.798154 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7edf2d9b-0d40-455e-855c-ad5c751a51ff service cinder] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:06:10.798603 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7edf2d9b-0d40-455e-855c-ad5c751a51ff service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:06:10.801727 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-7edf2d9b-0d40-455e-855c-ad5c751a51ff service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:06:10.816206 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf14219-043c-4b26-9110-f8a39da96f60 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:06:10.832524 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24cc1ad9-55ce-437b-a5ad-75cfe2cd39ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:06:10.834263 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24cc1ad9-55ce-437b-a5ad-75cfe2cd39ac admin admin] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:06:10.835896 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24cc1ad9-55ce-437b-a5ad-75cfe2cd39ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:06:10.836236 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24cc1ad9-55ce-437b-a5ad-75cfe2cd39ac admin admin] PATH_INFO: `/v3/projects` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:06:10.839019 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24cc1ad9-55ce-437b-a5ad-75cfe2cd39ac admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:06:10.843405 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24cc1ad9-55ce-437b-a5ad-75cfe2cd39ac admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:06:10.901872 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1062/2106] 10.222.0.40 () {60 vars in 1153 bytes} [Fri Aug 26 11:06:10 2022] POST /identity/v3/projects => generated 371 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 11:06:10.906155 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24cc1ad9-55ce-437b-a5ad-75cfe2cd39ac admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:06:10.913868 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7edf2d9b-0d40-455e-855c-ad5c751a51ff service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:06:10.916093 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7edf2d9b-0d40-455e-855c-ad5c751a51ff service cinder] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:06:10.933946 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1045/2107] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 11:06:10 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 179 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 11:06:11.023287 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24cc1ad9-55ce-437b-a5ad-75cfe2cd39ac admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:06:11.029680 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52da035-7246-45fa-980f-4a2bbecfc3d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:06:11.030633 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52da035-7246-45fa-980f-4a2bbecfc3d5 admin admin] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:06:11.031092 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52da035-7246-45fa-980f-4a2bbecfc3d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:06:11.031692 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52da035-7246-45fa-980f-4a2bbecfc3d5 admin admin] PATH_INFO: `/v3/users` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:06:11.033517 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c52da035-7246-45fa-980f-4a2bbecfc3d5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:06:11.037409 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c52da035-7246-45fa-980f-4a2bbecfc3d5 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:06:11.079851 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1063/2108] 10.222.0.40 () {60 vars in 1144 bytes} [Fri Aug 26 11:06:10 2022] POST /identity/v3/users => generated 286 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 11:06:11.085249 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7edf2d9b-0d40-455e-855c-ad5c751a51ff service cinder] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:06:11.098288 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7edf2d9b-0d40-455e-855c-ad5c751a51ff service cinder] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:06:11.103812 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4926ec3b-af82-4535-906a-c2930749d8e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:06:11.104458 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4926ec3b-af82-4535-906a-c2930749d8e6 admin admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:06:11.105017 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4926ec3b-af82-4535-906a-c2930749d8e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:06:11.105298 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4926ec3b-af82-4535-906a-c2930749d8e6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:06:11.106369 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4926ec3b-af82-4535-906a-c2930749d8e6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:06:11.107828 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4926ec3b-af82-4535-906a-c2930749d8e6 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:06:11.123718 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1046/2109] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 26 11:06:11 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 11:06:11.125037 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52da035-7246-45fa-980f-4a2bbecfc3d5 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:06:11.143572 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52da035-7246-45fa-980f-4a2bbecfc3d5 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:06:11.154420 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5010a8b8-55e1-4bfe-9a60-ef9d12ce5701 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:06:11.155311 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5010a8b8-55e1-4bfe-9a60-ef9d12ce5701 admin admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:06:11.156595 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5010a8b8-55e1-4bfe-9a60-ef9d12ce5701 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:06:11.157371 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5010a8b8-55e1-4bfe-9a60-ef9d12ce5701 admin admin] PATH_INFO: `/v3/roles` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:06:11.158166 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5010a8b8-55e1-4bfe-9a60-ef9d12ce5701 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:06:11.160302 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5010a8b8-55e1-4bfe-9a60-ef9d12ce5701 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:06:11.169454 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 11:06:11.183328 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4926ec3b-af82-4535-906a-c2930749d8e6 admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:06:11.186170 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1064/2110] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 26 11:06:11 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 26 11:06:11.206873 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4926ec3b-af82-4535-906a-c2930749d8e6 admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:06:11.213076 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85d858f-b673-458b-b4af-8d00743c3a77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:06:11.213828 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85d858f-b673-458b-b4af-8d00743c3a77 admin admin] REQUEST_METHOD: `PUT` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:06:11.214195 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85d858f-b673-458b-b4af-8d00743c3a77 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:06:11.214541 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85d858f-b673-458b-b4af-8d00743c3a77 admin admin] PATH_INFO: `/v3/projects/9a029ac2e8a446719206c416b51452c0/users/53665970ee784c6180c704d1a54c4c95/roles/23e174ce743b46eabe618c212691da55` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:06:11.247447 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f85d858f-b673-458b-b4af-8d00743c3a77 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9a029ac2e8a446719206c416b51452c0, user_id=53665970ee784c6180c704d1a54c4c95, role_id=23e174ce743b46eabe618c212691da55)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:06:11.250235 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f85d858f-b673-458b-b4af-8d00743c3a77 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:06:11.282242 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1047/2111] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 26 11:06:11 2022] PUT /identity/v3/projects/9a029ac2e8a446719206c416b51452c0/users/53665970ee784c6180c704d1a54c4c95/roles/23e174ce743b46eabe618c212691da55 => generated 0 bytes in 88 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 26 11:06:11.282843 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ddfb83d-ab27-47be-af95-6e1764992367 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:06:11.283129 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ddfb83d-ab27-47be-af95-6e1764992367 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:06:11.283432 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ddfb83d-ab27-47be-af95-6e1764992367 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:06:11.401512 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-4ddfb83d-ab27-47be-af95-6e1764992367 None None] MFA Rules not processed for user `53665970ee784c6180c704d1a54c4c95`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 11:06:11.403917 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-4ddfb83d-ab27-47be-af95-6e1764992367 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:06:11.536003 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1065/2112] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 11:06:11 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 250 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 11:06:11.546905 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85d858f-b673-458b-b4af-8d00743c3a77 admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:06:11.553480 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-f85d858f-b673-458b-b4af-8d00743c3a77 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:06:11.666194 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85d858f-b673-458b-b4af-8d00743c3a77 admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:06:11.670044 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d038b0ed-e817-43d1-8d31-7690c41bf037 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8631d7ecfd2849daa8694aecccd22a05', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e30ccf0c1204727b786876c141dbcd2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:06:11.670662 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d038b0ed-e817-43d1-8d31-7690c41bf037 service glance] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:06:11.671582 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d038b0ed-e817-43d1-8d31-7690c41bf037 service glance] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:06:11.671867 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d038b0ed-e817-43d1-8d31-7690c41bf037 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:06:11.681273 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d038b0ed-e817-43d1-8d31-7690c41bf037 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:06:11.682756 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d038b0ed-e817-43d1-8d31-7690c41bf037 service glance] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:06:11.694111 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1048/2113] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 11:06:11 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 11:06:13.526949 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ddfb83d-ab27-47be-af95-6e1764992367 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:06:13.528500 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-4ddfb83d-ab27-47be-af95-6e1764992367 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:06:13.644009 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ddfb83d-ab27-47be-af95-6e1764992367 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:06:13.650367 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9513db67-08ad-4009-9083-a5574ba164e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:06:13.651604 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9513db67-08ad-4009-9083-a5574ba164e5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:06:13.652709 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9513db67-08ad-4009-9083-a5574ba164e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:06:13.653147 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9513db67-08ad-4009-9083-a5574ba164e5 admin admin] PATH_INFO: `/v3/users/949a4c56bc6548ad9eb32e54d1f43ae7` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:06:13.654699 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9513db67-08ad-4009-9083-a5574ba164e5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=949a4c56bc6548ad9eb32e54d1f43ae7)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:06:13.656962 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9513db67-08ad-4009-9083-a5574ba164e5 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:06:13.709936 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555cf43e-2a19-456c-9d61-41e095b298bf None None] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:06:13.710932 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555cf43e-2a19-456c-9d61-41e095b298bf None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:06:13.711404 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555cf43e-2a19-456c-9d61-41e095b298bf None None] PATH_INFO: `/` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:06:13.713223 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1049/2114] 10.222.0.40 () {54 vars in 921 bytes} [Fri Aug 26 11:06:13 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 11:06:13.747761 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b25780-450e-4d49-a822-36bb34bedf89 None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:06:13.748363 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b25780-450e-4d49-a822-36bb34bedf89 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:06:13.748826 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b25780-450e-4d49-a822-36bb34bedf89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:06:13.756330 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-9513db67-08ad-4009-9083-a5574ba164e5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '949a4c56bc6548ad9eb32e54d1f43ae7'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:06:13.757288 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-9513db67-08ad-4009-9083-a5574ba164e5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '949a4c56bc6548ad9eb32e54d1f43ae7'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:06:13.775371 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-9513db67-08ad-4009-9083-a5574ba164e5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '949a4c56bc6548ad9eb32e54d1f43ae7'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:06:13.777440 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-9513db67-08ad-4009-9083-a5574ba164e5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '949a4c56bc6548ad9eb32e54d1f43ae7'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:06:13.799503 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-9513db67-08ad-4009-9083-a5574ba164e5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '949a4c56bc6548ad9eb32e54d1f43ae7'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:06:13.799674 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-74b25780-450e-4d49-a822-36bb34bedf89 None None] MFA Rules not processed for user `f29c5dfed31f42a9b5cbc0cd72e9fdc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 11:06:13.800784 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-74b25780-450e-4d49-a822-36bb34bedf89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:06:13.822866 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1066/2115] 10.222.0.40 () {58 vars in 1224 bytes} [Fri Aug 26 11:06:13 2022] DELETE /identity/v3/users/949a4c56bc6548ad9eb32e54d1f43ae7 => generated 0 bytes in 293 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 11:06:13.824164 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9513db67-08ad-4009-9083-a5574ba164e5 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:06:13.825738 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-9513db67-08ad-4009-9083-a5574ba164e5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:06:13.861778 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 11:06:13.898559 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 11:06:13.920887 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9513db67-08ad-4009-9083-a5574ba164e5 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:06:13.921159 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1050/2116] 10.222.0.40 () {58 vars in 1018 bytes} [Fri Aug 26 11:06:13 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 11:06:13.929774 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd429995-a68c-488c-9d8b-f43543ae906c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:06:13.930591 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd429995-a68c-488c-9d8b-f43543ae906c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:06:13.930896 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd429995-a68c-488c-9d8b-f43543ae906c admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:06:13.931294 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd429995-a68c-488c-9d8b-f43543ae906c admin admin] PATH_INFO: `/v3/users/d074367ea693406badce6f7b468f3147` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:06:13.932468 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd429995-a68c-488c-9d8b-f43543ae906c admin admin] RBAC: Authorizing `identity:delete_user(user_id=d074367ea693406badce6f7b468f3147)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:06:13.934414 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd429995-a68c-488c-9d8b-f43543ae906c admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:06:13.935450 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f856e09f-19e5-464c-913f-f7b548e6861f None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:06:13.935872 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f856e09f-19e5-464c-913f-f7b548e6861f None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:06:13.936338 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f856e09f-19e5-464c-913f-f7b548e6861f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:06:13.957186 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-f856e09f-19e5-464c-913f-f7b548e6861f None None] MFA Rules not processed for user `ca155a48e2024c32af3fd2e19e5fac11`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 11:06:13.958352 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-f856e09f-19e5-464c-913f-f7b548e6861f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:06:13.973213 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1051/2117] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 11:06:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 11:06:13.974168 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f856e09f-19e5-464c-913f-f7b548e6861f None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:06:13.975862 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-f856e09f-19e5-464c-913f-f7b548e6861f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:06:14.009846 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-bd429995-a68c-488c-9d8b-f43543ae906c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd074367ea693406badce6f7b468f3147'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:06:14.010680 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-bd429995-a68c-488c-9d8b-f43543ae906c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd074367ea693406badce6f7b468f3147'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:06:14.018511 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-bd429995-a68c-488c-9d8b-f43543ae906c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd074367ea693406badce6f7b468f3147'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:06:14.019188 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-bd429995-a68c-488c-9d8b-f43543ae906c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd074367ea693406badce6f7b468f3147'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:06:14.045889 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-bd429995-a68c-488c-9d8b-f43543ae906c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd074367ea693406badce6f7b468f3147'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:06:14.066405 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1067/2118] 10.222.0.40 () {58 vars in 1224 bytes} [Fri Aug 26 11:06:13 2022] DELETE /identity/v3/users/d074367ea693406badce6f7b468f3147 => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 11:06:14.069995 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd429995-a68c-488c-9d8b-f43543ae906c admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:06:14.069995 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-bd429995-a68c-488c-9d8b-f43543ae906c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:06:14.091555 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f856e09f-19e5-464c-913f-f7b548e6861f None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:06:14.147949 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd429995-a68c-488c-9d8b-f43543ae906c admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:06:14.152180 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e646ba9a-48c4-4126-9b4f-01e22d713512 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91da2acd2534423091537e1ba068d2b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e30ccf0c1204727b786876c141dbcd2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:06:14.152627 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e646ba9a-48c4-4126-9b4f-01e22d713512 service neutron] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:06:14.152987 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e646ba9a-48c4-4126-9b4f-01e22d713512 service neutron] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:06:14.153504 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e646ba9a-48c4-4126-9b4f-01e22d713512 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:06:14.155790 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-e646ba9a-48c4-4126-9b4f-01e22d713512 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:06:14.189242 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4267df3-331a-4970-bca1-ce338d3802b1 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f29c5dfed31f42a9b5cbc0cd72e9fdc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e30ccf0c1204727b786876c141dbcd2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:06:14.190261 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4267df3-331a-4970-bca1-ce338d3802b1 service placement] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:06:14.190930 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4267df3-331a-4970-bca1-ce338d3802b1 service placement] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:06:14.191340 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4267df3-331a-4970-bca1-ce338d3802b1 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:06:14.193500 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-c4267df3-331a-4970-bca1-ce338d3802b1 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:06:14.208056 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4267df3-331a-4970-bca1-ce338d3802b1 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:06:14.210470 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4267df3-331a-4970-bca1-ce338d3802b1 service placement] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:06:14.226190 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1052/2119] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 11:06:13 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 252 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 11:06:14.248298 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 11:06:14.286625 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e646ba9a-48c4-4126-9b4f-01e22d713512 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:06:14.288801 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e646ba9a-48c4-4126-9b4f-01e22d713512 service neutron] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:06:14.301380 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1068/2120] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 11:06:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 234 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 11:06:14.314575 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4267df3-331a-4970-bca1-ce338d3802b1 service placement] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:06:14.315998 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-c4267df3-331a-4970-bca1-ce338d3802b1 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:06:14.339955 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4267df3-331a-4970-bca1-ce338d3802b1 service placement] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:06:14.348095 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dddc7be8-1bf0-4d90-b7af-b9de68c8de47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:06:14.349357 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddc7be8-1bf0-4d90-b7af-b9de68c8de47 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:06:14.349357 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddc7be8-1bf0-4d90-b7af-b9de68c8de47 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:06:14.350642 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddc7be8-1bf0-4d90-b7af-b9de68c8de47 admin admin] PATH_INFO: `/v3/projects/6a00e426be1a41b4b37f7825869ac5cb` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:06:14.352878 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dddc7be8-1bf0-4d90-b7af-b9de68c8de47 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6a00e426be1a41b4b37f7825869ac5cb)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:06:14.357109 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dddc7be8-1bf0-4d90-b7af-b9de68c8de47 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:06:14.434257 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-dddc7be8-1bf0-4d90-b7af-b9de68c8de47 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6a00e426be1a41b4b37f7825869ac5cb', 'request_id': 'req-dddc7be8-1bf0-4d90-b7af-b9de68c8de47'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:06:14.442663 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-dddc7be8-1bf0-4d90-b7af-b9de68c8de47 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6a00e426be1a41b4b37f7825869ac5cb', 'request_id': 'req-dddc7be8-1bf0-4d90-b7af-b9de68c8de47'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:06:14.450362 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-dddc7be8-1bf0-4d90-b7af-b9de68c8de47 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6a00e426be1a41b4b37f7825869ac5cb', 'request_id': 'req-dddc7be8-1bf0-4d90-b7af-b9de68c8de47'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:06:14.453021 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 11:06:14.470472 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-dddc7be8-1bf0-4d90-b7af-b9de68c8de47 admin admin] The token cache is being invalidate because project 6a00e426be1a41b4b37f7825869ac5cb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81423) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 11:06:14.470472 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-dddc7be8-1bf0-4d90-b7af-b9de68c8de47 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:06:14.471294 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-dddc7be8-1bf0-4d90-b7af-b9de68c8de47 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:06:14.473764 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1053/2121] 10.222.0.40 () {58 vars in 1233 bytes} [Fri Aug 26 11:06:14 2022] DELETE /identity/v3/projects/6a00e426be1a41b4b37f7825869ac5cb => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 26 11:06:14.478728 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6cc0664-560e-4f78-a423-734602835be7 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:06:14.479040 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6cc0664-560e-4f78-a423-734602835be7 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:06:14.479393 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6cc0664-560e-4f78-a423-734602835be7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:06:14.509696 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-f6cc0664-560e-4f78-a423-734602835be7 None None] MFA Rules not processed for user `ca155a48e2024c32af3fd2e19e5fac11`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 11:06:14.511496 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-f6cc0664-560e-4f78-a423-734602835be7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:06:14.582168 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1069/2122] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 11:06:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 11:06:14.588951 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869f2745-2d5e-4692-a300-c1807c3a80b1 None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:06:14.589231 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869f2745-2d5e-4692-a300-c1807c3a80b1 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:06:14.589507 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869f2745-2d5e-4692-a300-c1807c3a80b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:06:14.618306 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-869f2745-2d5e-4692-a300-c1807c3a80b1 None None] MFA Rules not processed for user `ca155a48e2024c32af3fd2e19e5fac11`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 11:06:14.619745 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-869f2745-2d5e-4692-a300-c1807c3a80b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:06:14.633158 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1054/2123] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 11:06:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 11:06:14.635286 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6cc0664-560e-4f78-a423-734602835be7 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:06:14.637487 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-f6cc0664-560e-4f78-a423-734602835be7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:06:14.642132 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869f2745-2d5e-4692-a300-c1807c3a80b1 None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:06:14.662789 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6cc0664-560e-4f78-a423-734602835be7 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:06:14.673851 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869f2745-2d5e-4692-a300-c1807c3a80b1 None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:06:14.677029 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db948d8-9d7e-457c-84d5-152876b39ae6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:06:14.679089 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db948d8-9d7e-457c-84d5-152876b39ae6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:06:14.679946 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db948d8-9d7e-457c-84d5-152876b39ae6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:06:14.681298 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db948d8-9d7e-457c-84d5-152876b39ae6 admin admin] PATH_INFO: `/v3/projects/7025fe0928ec4955b13c1f185e72f9e0` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:06:14.682932 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db948d8-9d7e-457c-84d5-152876b39ae6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7025fe0928ec4955b13c1f185e72f9e0)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:06:14.686596 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db948d8-9d7e-457c-84d5-152876b39ae6 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:06:14.693498 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74aa7512-7d16-46c7-8105-3439eb188780 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:06:14.694191 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74aa7512-7d16-46c7-8105-3439eb188780 admin admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:06:14.694465 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74aa7512-7d16-46c7-8105-3439eb188780 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:06:14.694731 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74aa7512-7d16-46c7-8105-3439eb188780 admin admin] PATH_INFO: `/v3/domains` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:06:14.695387 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74aa7512-7d16-46c7-8105-3439eb188780 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 11:06:14.695683 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74aa7512-7d16-46c7-8105-3439eb188780 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:06:14.697270 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74aa7512-7d16-46c7-8105-3439eb188780 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:06:14.721076 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1055/2124] 10.222.0.40 () {58 vars in 1153 bytes} [Fri Aug 26 11:06:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 11:06:14.725725 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74aa7512-7d16-46c7-8105-3439eb188780 admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:06:14.751652 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74aa7512-7d16-46c7-8105-3439eb188780 admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:06:14.764340 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-6db948d8-9d7e-457c-84d5-152876b39ae6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7025fe0928ec4955b13c1f185e72f9e0', 'request_id': 'req-6db948d8-9d7e-457c-84d5-152876b39ae6'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:06:14.771921 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-6db948d8-9d7e-457c-84d5-152876b39ae6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7025fe0928ec4955b13c1f185e72f9e0', 'request_id': 'req-6db948d8-9d7e-457c-84d5-152876b39ae6'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:06:14.778781 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-6db948d8-9d7e-457c-84d5-152876b39ae6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7025fe0928ec4955b13c1f185e72f9e0', 'request_id': 'req-6db948d8-9d7e-457c-84d5-152876b39ae6'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:06:14.800552 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-6db948d8-9d7e-457c-84d5-152876b39ae6 admin admin] The token cache is being invalidate because project 7025fe0928ec4955b13c1f185e72f9e0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81424) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 11:06:14.800849 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-6db948d8-9d7e-457c-84d5-152876b39ae6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:06:14.803034 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-6db948d8-9d7e-457c-84d5-152876b39ae6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:06:14.814114 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1070/2125] 10.222.0.40 () {58 vars in 1233 bytes} [Fri Aug 26 11:06:14 2022] DELETE /identity/v3/projects/7025fe0928ec4955b13c1f185e72f9e0 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 11:06:14.872105 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6f8469-6ffc-46e9-80d3-6adbfb346244 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:06:14.872783 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6f8469-6ffc-46e9-80d3-6adbfb346244 admin admin] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:06:14.873156 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6f8469-6ffc-46e9-80d3-6adbfb346244 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:06:14.873582 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6f8469-6ffc-46e9-80d3-6adbfb346244 admin admin] PATH_INFO: `/v3/projects` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:06:14.874397 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea6f8469-6ffc-46e9-80d3-6adbfb346244 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:06:14.876882 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea6f8469-6ffc-46e9-80d3-6adbfb346244 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:06:14.917898 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1056/2126] 10.222.0.40 () {60 vars in 1153 bytes} [Fri Aug 26 11:06:14 2022] POST /identity/v3/projects => generated 405 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 26 11:06:14.921682 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db948d8-9d7e-457c-84d5-152876b39ae6 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:06:14.923199 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-6db948d8-9d7e-457c-84d5-152876b39ae6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:06:14.944859 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db948d8-9d7e-457c-84d5-152876b39ae6 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:06:15.030571 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6f8469-6ffc-46e9-80d3-6adbfb346244 admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:06:15.032770 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-ea6f8469-6ffc-46e9-80d3-6adbfb346244 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:06:15.070323 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461af22d-f529-4a2c-adc6-57452f3958d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:06:15.071324 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461af22d-f529-4a2c-adc6-57452f3958d7 admin admin] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:06:15.071766 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461af22d-f529-4a2c-adc6-57452f3958d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:06:15.072198 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461af22d-f529-4a2c-adc6-57452f3958d7 admin admin] PATH_INFO: `/v3/users` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:06:15.072825 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-461af22d-f529-4a2c-adc6-57452f3958d7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:06:15.075011 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-461af22d-f529-4a2c-adc6-57452f3958d7 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:06:15.086632 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 11:06:15.103341 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1071/2127] 10.222.0.40 () {60 vars in 1144 bytes} [Fri Aug 26 11:06:14 2022] POST /identity/v3/users => generated 303 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 11:06:15.107615 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461af22d-f529-4a2c-adc6-57452f3958d7 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:06:15.119552 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461af22d-f529-4a2c-adc6-57452f3958d7 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:06:15.124742 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76e9797-8269-49b8-aac7-10b37ac5dfaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:06:15.125465 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76e9797-8269-49b8-aac7-10b37ac5dfaf admin admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:06:15.125815 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76e9797-8269-49b8-aac7-10b37ac5dfaf admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:06:15.126171 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76e9797-8269-49b8-aac7-10b37ac5dfaf admin admin] PATH_INFO: `/v3/roles` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:06:15.126866 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e76e9797-8269-49b8-aac7-10b37ac5dfaf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:06:15.129271 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e76e9797-8269-49b8-aac7-10b37ac5dfaf admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:06:15.138448 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6f8469-6ffc-46e9-80d3-6adbfb346244 admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:06:15.141829 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1072/2128] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 26 11:06:15 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 26 11:06:15.145394 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e912901d-e75e-4ce2-b644-feacc139ddd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:06:15.145904 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76e9797-8269-49b8-aac7-10b37ac5dfaf admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:06:15.146793 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e912901d-e75e-4ce2-b644-feacc139ddd6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:06:15.148025 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e912901d-e75e-4ce2-b644-feacc139ddd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:06:15.148423 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e912901d-e75e-4ce2-b644-feacc139ddd6 admin admin] PATH_INFO: `/v3/users/28be7d3acfca4556b9b2769428815404` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:06:15.150317 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e912901d-e75e-4ce2-b644-feacc139ddd6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=28be7d3acfca4556b9b2769428815404)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:06:15.151864 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e912901d-e75e-4ce2-b644-feacc139ddd6 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:06:15.164175 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76e9797-8269-49b8-aac7-10b37ac5dfaf admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:06:15.173328 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c318a3-de36-4e25-9ff2-bc497254cb9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:06:15.174059 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c318a3-de36-4e25-9ff2-bc497254cb9f admin admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:06:15.174345 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c318a3-de36-4e25-9ff2-bc497254cb9f admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:06:15.174616 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c318a3-de36-4e25-9ff2-bc497254cb9f admin admin] PATH_INFO: `/v3/roles` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:06:15.175264 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c318a3-de36-4e25-9ff2-bc497254cb9f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:06:15.177208 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c318a3-de36-4e25-9ff2-bc497254cb9f admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:06:15.195338 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1073/2129] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 26 11:06:15 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 26 11:06:15.196052 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c318a3-de36-4e25-9ff2-bc497254cb9f admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:06:15.222213 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c318a3-de36-4e25-9ff2-bc497254cb9f admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:06:15.222693 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-e912901d-e75e-4ce2-b644-feacc139ddd6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '28be7d3acfca4556b9b2769428815404'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:06:15.231714 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-facf4275-5332-4ee1-9f2d-bbb5332e6df9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:06:15.232530 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-facf4275-5332-4ee1-9f2d-bbb5332e6df9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:06:15.233075 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-facf4275-5332-4ee1-9f2d-bbb5332e6df9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:06:15.233209 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-facf4275-5332-4ee1-9f2d-bbb5332e6df9 admin admin] PATH_INFO: `/v3/projects/7501ea71a2c144c89c8e77842c0adb3e/users/4e9238e0001244c8b90f364de2974218/roles/23e174ce743b46eabe618c212691da55` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:06:15.235901 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-e912901d-e75e-4ce2-b644-feacc139ddd6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '28be7d3acfca4556b9b2769428815404'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:06:15.236816 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-e912901d-e75e-4ce2-b644-feacc139ddd6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '28be7d3acfca4556b9b2769428815404'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:06:15.250916 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:DELETE=8,keystone:SELECT=91,keystone:INSERT=1 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 11:06:15.262299 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-e912901d-e75e-4ce2-b644-feacc139ddd6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '28be7d3acfca4556b9b2769428815404'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:06:15.264113 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-e912901d-e75e-4ce2-b644-feacc139ddd6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '28be7d3acfca4556b9b2769428815404'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:06:15.264727 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-facf4275-5332-4ee1-9f2d-bbb5332e6df9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7501ea71a2c144c89c8e77842c0adb3e, user_id=4e9238e0001244c8b90f364de2974218, role_id=23e174ce743b46eabe618c212691da55)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:06:15.271512 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-facf4275-5332-4ee1-9f2d-bbb5332e6df9 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:06:15.289507 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1074/2130] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 26 11:06:15 2022] PUT /identity/v3/projects/7501ea71a2c144c89c8e77842c0adb3e/users/4e9238e0001244c8b90f364de2974218/roles/23e174ce743b46eabe618c212691da55 => generated 0 bytes in 94 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 11:06:15.290605 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1057/2131] 10.222.0.40 () {58 vars in 1224 bytes} [Fri Aug 26 11:06:15 2022] DELETE /identity/v3/users/28be7d3acfca4556b9b2769428815404 => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 11:06:15.296537 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c4d850-6575-46b9-b820-ae95f0d21df2 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:06:15.299630 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c4d850-6575-46b9-b820-ae95f0d21df2 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:06:15.300183 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c4d850-6575-46b9-b820-ae95f0d21df2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:06:15.302300 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e3e362-3b90-4046-9643-d290e29844dd None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:06:15.303408 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e3e362-3b90-4046-9643-d290e29844dd None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:06:15.303774 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e3e362-3b90-4046-9643-d290e29844dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:06:15.327044 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-c5c4d850-6575-46b9-b820-ae95f0d21df2 None None] MFA Rules not processed for user `ca155a48e2024c32af3fd2e19e5fac11`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 11:06:15.328323 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-c5c4d850-6575-46b9-b820-ae95f0d21df2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:06:15.337164 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-95e3e362-3b90-4046-9643-d290e29844dd None None] MFA Rules not processed for user `4e9238e0001244c8b90f364de2974218`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 11:06:15.340416 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-95e3e362-3b90-4046-9643-d290e29844dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:06:15.415697 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1075/2132] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 11:06:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 11:06:15.442616 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1058/2133] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 11:06:15 2022] POST /identity/v3/auth/tokens => generated 3282 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 11:06:15.447964 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c4d850-6575-46b9-b820-ae95f0d21df2 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:06:15.450419 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-c5c4d850-6575-46b9-b820-ae95f0d21df2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:06:15.453053 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e3e362-3b90-4046-9643-d290e29844dd None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:06:15.456462 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-95e3e362-3b90-4046-9643-d290e29844dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:06:15.470422 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c4d850-6575-46b9-b820-ae95f0d21df2 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:06:15.478255 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3fadef-7251-4575-8bf7-48e6ef25b7a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:06:15.478965 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3fadef-7251-4575-8bf7-48e6ef25b7a8 admin admin] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:06:15.479959 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3fadef-7251-4575-8bf7-48e6ef25b7a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:06:15.480252 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3fadef-7251-4575-8bf7-48e6ef25b7a8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:06:15.481230 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d3fadef-7251-4575-8bf7-48e6ef25b7a8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:06:15.482916 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d3fadef-7251-4575-8bf7-48e6ef25b7a8 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:06:15.522915 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1076/2134] 10.222.0.40 () {60 vars in 1153 bytes} [Fri Aug 26 11:06:15 2022] POST /identity/v3/projects => generated 403 bytes in 75 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 11:06:15.527267 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3fadef-7251-4575-8bf7-48e6ef25b7a8 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:06:15.556825 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e3e362-3b90-4046-9643-d290e29844dd None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:06:15.615627 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3fadef-7251-4575-8bf7-48e6ef25b7a8 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:06:15.622127 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fdc6a3c-aea3-4f8d-8bc5-000107e40719 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:06:15.622799 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fdc6a3c-aea3-4f8d-8bc5-000107e40719 admin admin] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:06:15.623073 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fdc6a3c-aea3-4f8d-8bc5-000107e40719 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:06:15.623340 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fdc6a3c-aea3-4f8d-8bc5-000107e40719 admin admin] PATH_INFO: `/v3/users` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:06:15.623967 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fdc6a3c-aea3-4f8d-8bc5-000107e40719 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:06:15.626581 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fdc6a3c-aea3-4f8d-8bc5-000107e40719 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:06:15.645561 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c3033e-9d77-43b0-b699-dff1ccad386a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91da2acd2534423091537e1ba068d2b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e30ccf0c1204727b786876c141dbcd2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:06:15.647315 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c3033e-9d77-43b0-b699-dff1ccad386a service neutron] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:06:15.648045 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c3033e-9d77-43b0-b699-dff1ccad386a service neutron] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:06:15.648532 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c3033e-9d77-43b0-b699-dff1ccad386a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:06:15.658084 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1077/2135] 10.222.0.40 () {60 vars in 1144 bytes} [Fri Aug 26 11:06:15 2022] POST /identity/v3/users => generated 308 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 11:06:15.661650 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fdc6a3c-aea3-4f8d-8bc5-000107e40719 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:06:15.671469 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c3033e-9d77-43b0-b699-dff1ccad386a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:06:15.673633 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c3033e-9d77-43b0-b699-dff1ccad386a service neutron] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:06:15.675299 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fdc6a3c-aea3-4f8d-8bc5-000107e40719 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:06:15.683223 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adaddd87-40da-4c4e-9ff9-5ee59e89d179 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:06:15.683223 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adaddd87-40da-4c4e-9ff9-5ee59e89d179 admin admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:06:15.683223 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adaddd87-40da-4c4e-9ff9-5ee59e89d179 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:06:15.683565 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adaddd87-40da-4c4e-9ff9-5ee59e89d179 admin admin] PATH_INFO: `/v3/roles` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:06:15.684494 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adaddd87-40da-4c4e-9ff9-5ee59e89d179 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:06:15.686205 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adaddd87-40da-4c4e-9ff9-5ee59e89d179 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:06:15.695471 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1059/2136] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 11:06:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 243 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 26 11:06:15.702351 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1078/2137] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 26 11:06:15 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 26 11:06:15.703729 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c3033e-9d77-43b0-b699-dff1ccad386a service neutron] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:06:15.715497 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c3033e-9d77-43b0-b699-dff1ccad386a service neutron] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:06:15.721644 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03691145-71e2-44e3-a0b8-90f506870da9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:06:15.722337 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03691145-71e2-44e3-a0b8-90f506870da9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:06:15.722572 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03691145-71e2-44e3-a0b8-90f506870da9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:06:15.723502 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03691145-71e2-44e3-a0b8-90f506870da9 admin admin] PATH_INFO: `/v3/projects/9e232a2c10f0481ab44a03cacbd27845/users/0602cf79916d42c0ba0e7bd303b5e45d/roles/1f97e4602e2e4d6498913a81f9ae8d6a` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:06:15.742245 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03691145-71e2-44e3-a0b8-90f506870da9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9e232a2c10f0481ab44a03cacbd27845, user_id=0602cf79916d42c0ba0e7bd303b5e45d, role_id=1f97e4602e2e4d6498913a81f9ae8d6a)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:06:15.757796 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03691145-71e2-44e3-a0b8-90f506870da9 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:06:15.765916 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1060/2138] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 26 11:06:15 2022] PUT /identity/v3/projects/9e232a2c10f0481ab44a03cacbd27845/users/0602cf79916d42c0ba0e7bd303b5e45d/roles/1f97e4602e2e4d6498913a81f9ae8d6a => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 26 11:06:15.769678 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adaddd87-40da-4c4e-9ff9-5ee59e89d179 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:06:15.776247 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:INSERT=7,keystone:SELECT=93 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 11:06:15.781844 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adaddd87-40da-4c4e-9ff9-5ee59e89d179 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:06:15.879667 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3314754-e886-4f96-be68-19d7ecf30f19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:06:15.880737 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3314754-e886-4f96-be68-19d7ecf30f19 admin admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:06:15.880737 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3314754-e886-4f96-be68-19d7ecf30f19 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:06:15.881354 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3314754-e886-4f96-be68-19d7ecf30f19 admin admin] PATH_INFO: `/v3/roles` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:06:15.882131 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3314754-e886-4f96-be68-19d7ecf30f19 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:06:15.884035 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3314754-e886-4f96-be68-19d7ecf30f19 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:06:15.900912 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1079/2139] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 26 11:06:15 2022] GET /identity/v3/roles => generated 1420 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 11:06:15.906098 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03691145-71e2-44e3-a0b8-90f506870da9 admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:06:15.912467 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:DELETE=3,keystone:SELECT=95,keystone:INSERT=2 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 11:06:15.923654 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03691145-71e2-44e3-a0b8-90f506870da9 admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:06:15.935186 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02027706-2a3d-4a52-a713-711af0e2aee7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:06:15.935965 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02027706-2a3d-4a52-a713-711af0e2aee7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:06:15.936426 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02027706-2a3d-4a52-a713-711af0e2aee7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:06:15.936889 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02027706-2a3d-4a52-a713-711af0e2aee7 admin admin] PATH_INFO: `/v3/projects/9e232a2c10f0481ab44a03cacbd27845/users/0602cf79916d42c0ba0e7bd303b5e45d/roles/1f97e4602e2e4d6498913a81f9ae8d6a` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:06:15.939076 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02027706-2a3d-4a52-a713-711af0e2aee7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9e232a2c10f0481ab44a03cacbd27845, user_id=0602cf79916d42c0ba0e7bd303b5e45d, role_id=1f97e4602e2e4d6498913a81f9ae8d6a)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:06:15.941998 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02027706-2a3d-4a52-a713-711af0e2aee7 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:06:15.955559 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1061/2140] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 26 11:06:15 2022] PUT /identity/v3/projects/9e232a2c10f0481ab44a03cacbd27845/users/0602cf79916d42c0ba0e7bd303b5e45d/roles/1f97e4602e2e4d6498913a81f9ae8d6a => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 11:06:15.967566 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486953ff-02bf-48c2-8320-db92fd3bbc3f None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:06:15.968287 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486953ff-02bf-48c2-8320-db92fd3bbc3f None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:06:15.968287 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486953ff-02bf-48c2-8320-db92fd3bbc3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:06:16.000635 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-486953ff-02bf-48c2-8320-db92fd3bbc3f None None] MFA Rules not processed for user `0602cf79916d42c0ba0e7bd303b5e45d`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 11:06:16.001958 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-486953ff-02bf-48c2-8320-db92fd3bbc3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:06:16.095063 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1080/2141] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 11:06:15 2022] POST /identity/v3/auth/tokens => generated 3347 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 11:06:16.107049 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02027706-2a3d-4a52-a713-711af0e2aee7 admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:06:16.110342 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-02027706-2a3d-4a52-a713-711af0e2aee7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:06:16.136745 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486953ff-02bf-48c2-8320-db92fd3bbc3f None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:06:16.138310 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-486953ff-02bf-48c2-8320-db92fd3bbc3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:06:16.203694 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02027706-2a3d-4a52-a713-711af0e2aee7 admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:06:16.207691 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e597052a-e3dd-452f-8e9b-ce9bdfd98d1f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c277338a7f814cc6b01c5419775031bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e30ccf0c1204727b786876c141dbcd2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:06:16.208292 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e597052a-e3dd-452f-8e9b-ce9bdfd98d1f service cinder] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:06:16.208653 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e597052a-e3dd-452f-8e9b-ce9bdfd98d1f service cinder] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:06:16.209605 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e597052a-e3dd-452f-8e9b-ce9bdfd98d1f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:06:16.222491 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e597052a-e3dd-452f-8e9b-ce9bdfd98d1f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:06:16.223980 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e597052a-e3dd-452f-8e9b-ce9bdfd98d1f service cinder] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:06:16.242315 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486953ff-02bf-48c2-8320-db92fd3bbc3f None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:06:16.244583 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1062/2142] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 11:06:16 2022] GET /identity/v3/auth/tokens => generated 3347 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 26 11:06:16.247592 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f47bbbf-405e-4035-a98f-90eee4006318 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:06:16.248924 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f47bbbf-405e-4035-a98f-90eee4006318 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:06:16.248924 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f47bbbf-405e-4035-a98f-90eee4006318 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:06:16.248924 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f47bbbf-405e-4035-a98f-90eee4006318 admin admin] PATH_INFO: `/v3/projects/178703c6da2441a2bf91d5d855302d9c` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:06:16.250057 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f47bbbf-405e-4035-a98f-90eee4006318 admin admin] RBAC: Authorizing `identity:delete_project(project_id=178703c6da2441a2bf91d5d855302d9c)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:06:16.251986 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f47bbbf-405e-4035-a98f-90eee4006318 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:06:16.303414 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 11:06:16.332269 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-9f47bbbf-405e-4035-a98f-90eee4006318 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '178703c6da2441a2bf91d5d855302d9c', 'request_id': 'req-9f47bbbf-405e-4035-a98f-90eee4006318'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:06:16.342000 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-9f47bbbf-405e-4035-a98f-90eee4006318 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '178703c6da2441a2bf91d5d855302d9c', 'request_id': 'req-9f47bbbf-405e-4035-a98f-90eee4006318'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:06:16.350883 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-9f47bbbf-405e-4035-a98f-90eee4006318 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '178703c6da2441a2bf91d5d855302d9c', 'request_id': 'req-9f47bbbf-405e-4035-a98f-90eee4006318'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:06:16.375650 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-9f47bbbf-405e-4035-a98f-90eee4006318 admin admin] The token cache is being invalidate because project 178703c6da2441a2bf91d5d855302d9c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81424) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 11:06:16.376387 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-9f47bbbf-405e-4035-a98f-90eee4006318 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:06:16.377445 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-9f47bbbf-405e-4035-a98f-90eee4006318 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:06:16.379892 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1081/2143] 10.222.0.40 () {58 vars in 1233 bytes} [Fri Aug 26 11:06:16 2022] DELETE /identity/v3/projects/178703c6da2441a2bf91d5d855302d9c => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 11:06:16.384944 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e597052a-e3dd-452f-8e9b-ce9bdfd98d1f service cinder] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:06:16.400324 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e597052a-e3dd-452f-8e9b-ce9bdfd98d1f service cinder] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:06:16.451846 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f47bbbf-405e-4035-a98f-90eee4006318 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:06:16.453607 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-9f47bbbf-405e-4035-a98f-90eee4006318 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:06:16.476551 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2d7461-c8bb-4117-8ec5-6ad3a7e19261 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:06:16.477839 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2d7461-c8bb-4117-8ec5-6ad3a7e19261 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:06:16.478247 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2d7461-c8bb-4117-8ec5-6ad3a7e19261 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:06:16.478637 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2d7461-c8bb-4117-8ec5-6ad3a7e19261 admin admin] PATH_INFO: `/v3/users/75052b4fe894452e97d03444bf26641e` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:06:16.480268 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a2d7461-c8bb-4117-8ec5-6ad3a7e19261 admin admin] RBAC: Authorizing `identity:delete_user(user_id=75052b4fe894452e97d03444bf26641e)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:06:16.482347 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a2d7461-c8bb-4117-8ec5-6ad3a7e19261 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:06:16.545432 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f47bbbf-405e-4035-a98f-90eee4006318 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:06:16.550201 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6a6af0-076c-4472-ae16-cfcc7392b1cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:06:16.550836 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6a6af0-076c-4472-ae16-cfcc7392b1cf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:06:16.551113 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6a6af0-076c-4472-ae16-cfcc7392b1cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:06:16.551400 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6a6af0-076c-4472-ae16-cfcc7392b1cf admin admin] PATH_INFO: `/v3/users/4e9238e0001244c8b90f364de2974218` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:06:16.553067 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa6a6af0-076c-4472-ae16-cfcc7392b1cf admin admin] RBAC: Authorizing `identity:delete_user(user_id=4e9238e0001244c8b90f364de2974218)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:06:16.555111 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa6a6af0-076c-4472-ae16-cfcc7392b1cf admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:06:16.559084 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-5a2d7461-c8bb-4117-8ec5-6ad3a7e19261 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '75052b4fe894452e97d03444bf26641e'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:06:16.566347 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-5a2d7461-c8bb-4117-8ec5-6ad3a7e19261 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '75052b4fe894452e97d03444bf26641e'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:06:16.567052 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-5a2d7461-c8bb-4117-8ec5-6ad3a7e19261 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '75052b4fe894452e97d03444bf26641e'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:06:16.580968 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-5a2d7461-c8bb-4117-8ec5-6ad3a7e19261 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '75052b4fe894452e97d03444bf26641e'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:06:16.581668 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-5a2d7461-c8bb-4117-8ec5-6ad3a7e19261 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '75052b4fe894452e97d03444bf26641e'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:06:16.602899 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1063/2144] 10.222.0.40 () {58 vars in 1224 bytes} [Fri Aug 26 11:06:16 2022] DELETE /identity/v3/users/75052b4fe894452e97d03444bf26641e => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 11:06:16.618768 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-fa6a6af0-076c-4472-ae16-cfcc7392b1cf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4e9238e0001244c8b90f364de2974218'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:06:16.619667 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-fa6a6af0-076c-4472-ae16-cfcc7392b1cf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4e9238e0001244c8b90f364de2974218'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:06:16.627531 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-fa6a6af0-076c-4472-ae16-cfcc7392b1cf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4e9238e0001244c8b90f364de2974218'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:06:16.628350 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-fa6a6af0-076c-4472-ae16-cfcc7392b1cf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4e9238e0001244c8b90f364de2974218'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:06:16.643289 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-fa6a6af0-076c-4472-ae16-cfcc7392b1cf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4e9238e0001244c8b90f364de2974218'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:06:16.663407 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1082/2145] 10.222.0.40 () {58 vars in 1224 bytes} [Fri Aug 26 11:06:16 2022] DELETE /identity/v3/users/4e9238e0001244c8b90f364de2974218 => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 11:06:16.668381 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d7e94f-52b2-496d-95df-e629021e49a9 None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:06:16.668871 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d7e94f-52b2-496d-95df-e629021e49a9 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:06:16.669164 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d7e94f-52b2-496d-95df-e629021e49a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:06:16.695456 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-74d7e94f-52b2-496d-95df-e629021e49a9 None None] MFA Rules not processed for user `ca155a48e2024c32af3fd2e19e5fac11`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 11:06:16.696669 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-74d7e94f-52b2-496d-95df-e629021e49a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:06:16.703092 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 11:06:16.714715 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe44f633-36ba-4132-a389-4e349175b675 None None] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:06:16.715036 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe44f633-36ba-4132-a389-4e349175b675 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:06:16.715648 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe44f633-36ba-4132-a389-4e349175b675 None None] PATH_INFO: `/` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:06:16.717102 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1083/2146] 10.222.0.40 () {54 vars in 921 bytes} [Fri Aug 26 11:06:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 11:06:16.749226 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9df9a94-9668-4200-98bc-099007ae9e26 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:06:16.749719 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9df9a94-9668-4200-98bc-099007ae9e26 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:06:16.750492 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9df9a94-9668-4200-98bc-099007ae9e26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:06:16.795404 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-c9df9a94-9668-4200-98bc-099007ae9e26 None None] MFA Rules not processed for user `f29c5dfed31f42a9b5cbc0cd72e9fdc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 11:06:16.798926 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1064/2147] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 11:06:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 11:06:16.799403 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-c9df9a94-9668-4200-98bc-099007ae9e26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:06:16.820715 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d7e94f-52b2-496d-95df-e629021e49a9 None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:06:16.822354 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-74d7e94f-52b2-496d-95df-e629021e49a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:06:16.837690 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d7e94f-52b2-496d-95df-e629021e49a9 None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:06:16.843718 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f1d52e-e4bb-4d38-868a-899601990dec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:06:16.844070 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f1d52e-e4bb-4d38-868a-899601990dec admin admin] REQUEST_METHOD: `DELETE` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:06:16.844344 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f1d52e-e4bb-4d38-868a-899601990dec admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:06:16.844612 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f1d52e-e4bb-4d38-868a-899601990dec admin admin] PATH_INFO: `/v3/users/d9b0982290b346ac9967bf57ab5cdba1` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:06:16.845858 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f1d52e-e4bb-4d38-868a-899601990dec admin admin] RBAC: Authorizing `identity:delete_user(user_id=d9b0982290b346ac9967bf57ab5cdba1)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:06:16.847739 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f1d52e-e4bb-4d38-868a-899601990dec admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:06:16.849049 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:DELETE=16,keystone:SELECT=82,keystone:INSERT=2 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 11:06:16.892959 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1084/2148] 10.222.0.40 () {58 vars in 1018 bytes} [Fri Aug 26 11:06:16 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 11:06:16.894073 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9df9a94-9668-4200-98bc-099007ae9e26 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:06:16.898500 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-c9df9a94-9668-4200-98bc-099007ae9e26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:06:16.930692 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-46f1d52e-e4bb-4d38-868a-899601990dec admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd9b0982290b346ac9967bf57ab5cdba1'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:06:16.938545 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-46f1d52e-e4bb-4d38-868a-899601990dec admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd9b0982290b346ac9967bf57ab5cdba1'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:06:16.939346 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-46f1d52e-e4bb-4d38-868a-899601990dec admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd9b0982290b346ac9967bf57ab5cdba1'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:06:16.952876 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-46f1d52e-e4bb-4d38-868a-899601990dec admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd9b0982290b346ac9967bf57ab5cdba1'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:06:16.953663 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-46f1d52e-e4bb-4d38-868a-899601990dec admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd9b0982290b346ac9967bf57ab5cdba1'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:06:16.969061 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9df9a94-9668-4200-98bc-099007ae9e26 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:06:16.969811 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1065/2149] 10.222.0.40 () {58 vars in 1224 bytes} [Fri Aug 26 11:06:16 2022] DELETE /identity/v3/users/d9b0982290b346ac9967bf57ab5cdba1 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 26 11:06:16.973125 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f1d52e-e4bb-4d38-868a-899601990dec admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:06:16.973125 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-46f1d52e-e4bb-4d38-868a-899601990dec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:06:17.040864 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e668d36-ce24-4715-9cd7-a877450a3e58 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91da2acd2534423091537e1ba068d2b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e30ccf0c1204727b786876c141dbcd2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:06:17.041494 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e668d36-ce24-4715-9cd7-a877450a3e58 service neutron] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:06:17.042034 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e668d36-ce24-4715-9cd7-a877450a3e58 service neutron] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:06:17.042165 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e668d36-ce24-4715-9cd7-a877450a3e58 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:06:17.043854 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-3e668d36-ce24-4715-9cd7-a877450a3e58 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:06:17.087005 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f1d52e-e4bb-4d38-868a-899601990dec admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:06:17.092767 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3afab96-6ffc-4074-8de9-863a1210b39f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f29c5dfed31f42a9b5cbc0cd72e9fdc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e30ccf0c1204727b786876c141dbcd2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:06:17.093389 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3afab96-6ffc-4074-8de9-863a1210b39f service placement] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:06:17.093667 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3afab96-6ffc-4074-8de9-863a1210b39f service placement] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:06:17.093940 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3afab96-6ffc-4074-8de9-863a1210b39f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:06:17.095806 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-e3afab96-6ffc-4074-8de9-863a1210b39f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:06:17.111398 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3afab96-6ffc-4074-8de9-863a1210b39f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:06:17.113130 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3afab96-6ffc-4074-8de9-863a1210b39f service placement] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:06:17.125625 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1066/2150] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 11:06:16 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 11:06:17.126498 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3afab96-6ffc-4074-8de9-863a1210b39f service placement] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:06:17.128174 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-e3afab96-6ffc-4074-8de9-863a1210b39f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:06:17.134435 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e668d36-ce24-4715-9cd7-a877450a3e58 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:06:17.136349 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e668d36-ce24-4715-9cd7-a877450a3e58 service neutron] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:06:17.143677 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3afab96-6ffc-4074-8de9-863a1210b39f service placement] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:06:17.149587 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9269973e-cc93-4637-8cc4-6f32e25e60fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:06:17.150301 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9269973e-cc93-4637-8cc4-6f32e25e60fb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:06:17.150584 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9269973e-cc93-4637-8cc4-6f32e25e60fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:06:17.151395 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9269973e-cc93-4637-8cc4-6f32e25e60fb admin admin] PATH_INFO: `/v3/projects/e8a9b964e5f44287a0c3825eeb06e042` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:06:17.152221 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9269973e-cc93-4637-8cc4-6f32e25e60fb admin admin] RBAC: Authorizing `identity:delete_project(project_id=e8a9b964e5f44287a0c3825eeb06e042)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:06:17.154412 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9269973e-cc93-4637-8cc4-6f32e25e60fb admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:06:17.156493 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1085/2151] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 11:06:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 263 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 11:06:17.164735 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 11:06:17.214119 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-9269973e-cc93-4637-8cc4-6f32e25e60fb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e8a9b964e5f44287a0c3825eeb06e042', 'request_id': 'req-9269973e-cc93-4637-8cc4-6f32e25e60fb'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:06:17.222643 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-9269973e-cc93-4637-8cc4-6f32e25e60fb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e8a9b964e5f44287a0c3825eeb06e042', 'request_id': 'req-9269973e-cc93-4637-8cc4-6f32e25e60fb'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:06:17.232412 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-9269973e-cc93-4637-8cc4-6f32e25e60fb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e8a9b964e5f44287a0c3825eeb06e042', 'request_id': 'req-9269973e-cc93-4637-8cc4-6f32e25e60fb'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:06:17.235912 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e668d36-ce24-4715-9cd7-a877450a3e58 service neutron] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:06:17.237729 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-3e668d36-ce24-4715-9cd7-a877450a3e58 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:06:17.250827 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-9269973e-cc93-4637-8cc4-6f32e25e60fb admin admin] The token cache is being invalidate because project e8a9b964e5f44287a0c3825eeb06e042 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81423) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 11:06:17.251373 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e668d36-ce24-4715-9cd7-a877450a3e58 service neutron] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:06:17.251716 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-9269973e-cc93-4637-8cc4-6f32e25e60fb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:06:17.252582 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-9269973e-cc93-4637-8cc4-6f32e25e60fb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:06:17.254354 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1067/2152] 10.222.0.40 () {58 vars in 1233 bytes} [Fri Aug 26 11:06:17 2022] DELETE /identity/v3/projects/e8a9b964e5f44287a0c3825eeb06e042 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 11:06:17.259498 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa3c137-44c8-40e5-ba42-ec2174300bec None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:06:17.259736 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa3c137-44c8-40e5-ba42-ec2174300bec None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:06:17.260059 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa3c137-44c8-40e5-ba42-ec2174300bec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:06:17.295524 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-3aa3c137-44c8-40e5-ba42-ec2174300bec None None] MFA Rules not processed for user `ca155a48e2024c32af3fd2e19e5fac11`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 11:06:17.296952 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-3aa3c137-44c8-40e5-ba42-ec2174300bec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:06:17.311270 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 11:06:17.345794 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f68e01a-a76b-40f2-926a-02840c6f3d15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:06:17.346484 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f68e01a-a76b-40f2-926a-02840c6f3d15 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:06:17.346759 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f68e01a-a76b-40f2-926a-02840c6f3d15 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:06:17.347325 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f68e01a-a76b-40f2-926a-02840c6f3d15 admin admin] PATH_INFO: `/v3/projects/48d502e519f4425a8602bc040762fe47` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:06:17.348214 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f68e01a-a76b-40f2-926a-02840c6f3d15 admin admin] RBAC: Authorizing `identity:delete_project(project_id=48d502e519f4425a8602bc040762fe47)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:06:17.350321 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f68e01a-a76b-40f2-926a-02840c6f3d15 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:06:17.383947 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1068/2153] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 11:06:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 11:06:17.392669 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b194d65-449b-484c-9f00-cb2fec89f952 None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:06:17.392951 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b194d65-449b-484c-9f00-cb2fec89f952 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:06:17.393222 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b194d65-449b-484c-9f00-cb2fec89f952 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:06:17.406829 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-7f68e01a-a76b-40f2-926a-02840c6f3d15 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '48d502e519f4425a8602bc040762fe47', 'request_id': 'req-7f68e01a-a76b-40f2-926a-02840c6f3d15'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:06:17.418391 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-6b194d65-449b-484c-9f00-cb2fec89f952 None None] MFA Rules not processed for user `ca155a48e2024c32af3fd2e19e5fac11`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 11:06:17.418685 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-6b194d65-449b-484c-9f00-cb2fec89f952 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:06:17.419855 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-7f68e01a-a76b-40f2-926a-02840c6f3d15 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '48d502e519f4425a8602bc040762fe47', 'request_id': 'req-7f68e01a-a76b-40f2-926a-02840c6f3d15'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:06:17.441630 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-7f68e01a-a76b-40f2-926a-02840c6f3d15 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '48d502e519f4425a8602bc040762fe47', 'request_id': 'req-7f68e01a-a76b-40f2-926a-02840c6f3d15'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:06:17.457639 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-7f68e01a-a76b-40f2-926a-02840c6f3d15 admin admin] The token cache is being invalidate because project 48d502e519f4425a8602bc040762fe47 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81424) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 11:06:17.457952 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-7f68e01a-a76b-40f2-926a-02840c6f3d15 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:06:17.458773 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-7f68e01a-a76b-40f2-926a-02840c6f3d15 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:06:17.462054 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1086/2154] 10.222.0.40 () {58 vars in 1233 bytes} [Fri Aug 26 11:06:17 2022] DELETE /identity/v3/projects/48d502e519f4425a8602bc040762fe47 => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 11:06:17.465001 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f68e01a-a76b-40f2-926a-02840c6f3d15 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:06:17.466641 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-7f68e01a-a76b-40f2-926a-02840c6f3d15 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:06:17.492357 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1069/2155] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 11:06:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 11:06:17.499261 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b194d65-449b-484c-9f00-cb2fec89f952 None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:06:17.509337 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b194d65-449b-484c-9f00-cb2fec89f952 None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:06:17.515048 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63cd1f93-9ded-402b-8419-f993b2b94ab8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:06:17.518410 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63cd1f93-9ded-402b-8419-f993b2b94ab8 admin admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:06:17.518410 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63cd1f93-9ded-402b-8419-f993b2b94ab8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:06:17.518410 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63cd1f93-9ded-402b-8419-f993b2b94ab8 admin admin] PATH_INFO: `/v3/domains` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:06:17.518410 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63cd1f93-9ded-402b-8419-f993b2b94ab8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 11:06:17.518410 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63cd1f93-9ded-402b-8419-f993b2b94ab8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:06:17.518984 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63cd1f93-9ded-402b-8419-f993b2b94ab8 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:06:17.531833 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1070/2156] 10.222.0.40 () {58 vars in 1153 bytes} [Fri Aug 26 11:06:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 11:06:17.532595 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63cd1f93-9ded-402b-8419-f993b2b94ab8 admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:06:17.535623 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-63cd1f93-9ded-402b-8419-f993b2b94ab8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:06:17.544542 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f68e01a-a76b-40f2-926a-02840c6f3d15 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:06:17.550281 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e0cbb6-cb09-403f-864d-845f85021630 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:06:17.550503 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63cd1f93-9ded-402b-8419-f993b2b94ab8 admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:06:17.551440 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e0cbb6-cb09-403f-864d-845f85021630 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:06:17.551932 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e0cbb6-cb09-403f-864d-845f85021630 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:06:17.553052 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e0cbb6-cb09-403f-864d-845f85021630 admin admin] PATH_INFO: `/v3/users/5110464c90a64882b18292c370097798` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:06:17.554894 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e0cbb6-cb09-403f-864d-845f85021630 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5110464c90a64882b18292c370097798)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:06:17.555730 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c8e155-fec0-4601-a2b6-979cb7b34c8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:06:17.556354 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c8e155-fec0-4601-a2b6-979cb7b34c8e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:06:17.556630 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c8e155-fec0-4601-a2b6-979cb7b34c8e admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:06:17.556908 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c8e155-fec0-4601-a2b6-979cb7b34c8e admin admin] PATH_INFO: `/v3/projects/7501ea71a2c144c89c8e77842c0adb3e` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:06:17.557171 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e0cbb6-cb09-403f-864d-845f85021630 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:06:17.558160 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c8e155-fec0-4601-a2b6-979cb7b34c8e admin admin] RBAC: Authorizing `identity:delete_project(project_id=7501ea71a2c144c89c8e77842c0adb3e)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:06:17.561050 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c8e155-fec0-4601-a2b6-979cb7b34c8e admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:06:17.584847 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 11:06:17.610125 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-47e0cbb6-cb09-403f-864d-845f85021630 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5110464c90a64882b18292c370097798'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:06:17.610795 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-47e0cbb6-cb09-403f-864d-845f85021630 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5110464c90a64882b18292c370097798'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:06:17.612547 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-34c8e155-fec0-4601-a2b6-979cb7b34c8e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7501ea71a2c144c89c8e77842c0adb3e', 'request_id': 'req-34c8e155-fec0-4601-a2b6-979cb7b34c8e'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:06:17.619387 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-47e0cbb6-cb09-403f-864d-845f85021630 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5110464c90a64882b18292c370097798'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:06:17.620257 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-47e0cbb6-cb09-403f-864d-845f85021630 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5110464c90a64882b18292c370097798'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:06:17.620596 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-34c8e155-fec0-4601-a2b6-979cb7b34c8e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7501ea71a2c144c89c8e77842c0adb3e', 'request_id': 'req-34c8e155-fec0-4601-a2b6-979cb7b34c8e'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:06:17.626204 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-34c8e155-fec0-4601-a2b6-979cb7b34c8e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7501ea71a2c144c89c8e77842c0adb3e', 'request_id': 'req-34c8e155-fec0-4601-a2b6-979cb7b34c8e'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:06:17.628124 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 11:06:17.635103 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-47e0cbb6-cb09-403f-864d-845f85021630 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5110464c90a64882b18292c370097798'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:06:17.645072 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-34c8e155-fec0-4601-a2b6-979cb7b34c8e admin admin] The token cache is being invalidate because project 7501ea71a2c144c89c8e77842c0adb3e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81423) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 11:06:17.645343 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-34c8e155-fec0-4601-a2b6-979cb7b34c8e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:06:17.646688 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-34c8e155-fec0-4601-a2b6-979cb7b34c8e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:06:17.648273 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1071/2157] 10.222.0.40 () {58 vars in 1233 bytes} [Fri Aug 26 11:06:17 2022] DELETE /identity/v3/projects/7501ea71a2c144c89c8e77842c0adb3e => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 11:06:17.651231 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c8e155-fec0-4601-a2b6-979cb7b34c8e admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:06:17.651231 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-34c8e155-fec0-4601-a2b6-979cb7b34c8e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:06:17.652450 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1087/2158] 10.222.0.40 () {58 vars in 1224 bytes} [Fri Aug 26 11:06:17 2022] DELETE /identity/v3/users/5110464c90a64882b18292c370097798 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 11:06:17.653341 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e0cbb6-cb09-403f-864d-845f85021630 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:06:17.654699 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-47e0cbb6-cb09-403f-864d-845f85021630 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:06:17.735242 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e0cbb6-cb09-403f-864d-845f85021630 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:06:17.736217 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c8e155-fec0-4601-a2b6-979cb7b34c8e admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:06:17.740754 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282e00ae-b936-44f1-8c7e-24527237e9a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:06:17.741390 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282e00ae-b936-44f1-8c7e-24527237e9a2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:06:17.741632 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7747003c-49a3-4636-ae26-d9b890e28f29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:06:17.741800 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282e00ae-b936-44f1-8c7e-24527237e9a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:06:17.741929 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282e00ae-b936-44f1-8c7e-24527237e9a2 admin admin] PATH_INFO: `/v3/users/0602cf79916d42c0ba0e7bd303b5e45d` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:06:17.742265 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7747003c-49a3-4636-ae26-d9b890e28f29 admin admin] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:06:17.743047 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7747003c-49a3-4636-ae26-d9b890e28f29 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:06:17.743327 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7747003c-49a3-4636-ae26-d9b890e28f29 admin admin] PATH_INFO: `/v3/projects` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:06:17.743442 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-282e00ae-b936-44f1-8c7e-24527237e9a2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0602cf79916d42c0ba0e7bd303b5e45d)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:06:17.743952 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7747003c-49a3-4636-ae26-d9b890e28f29 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:06:17.745349 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-282e00ae-b936-44f1-8c7e-24527237e9a2 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:06:17.745837 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7747003c-49a3-4636-ae26-d9b890e28f29 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:06:17.779688 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1072/2159] 10.222.0.40 () {60 vars in 1153 bytes} [Fri Aug 26 11:06:17 2022] POST /identity/v3/projects => generated 381 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 11:06:17.780788 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7747003c-49a3-4636-ae26-d9b890e28f29 admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:06:17.829070 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-282e00ae-b936-44f1-8c7e-24527237e9a2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0602cf79916d42c0ba0e7bd303b5e45d'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:06:17.829754 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-282e00ae-b936-44f1-8c7e-24527237e9a2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0602cf79916d42c0ba0e7bd303b5e45d'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:06:17.843127 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-282e00ae-b936-44f1-8c7e-24527237e9a2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0602cf79916d42c0ba0e7bd303b5e45d'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:06:17.844263 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-282e00ae-b936-44f1-8c7e-24527237e9a2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0602cf79916d42c0ba0e7bd303b5e45d'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:06:17.877188 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7747003c-49a3-4636-ae26-d9b890e28f29 admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:06:17.882853 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb13c529-1836-480c-9037-9a3d61dc5463 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:06:17.883598 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb13c529-1836-480c-9037-9a3d61dc5463 admin admin] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:06:17.883977 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb13c529-1836-480c-9037-9a3d61dc5463 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:06:17.884348 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb13c529-1836-480c-9037-9a3d61dc5463 admin admin] PATH_INFO: `/v3/users` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:06:17.885070 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb13c529-1836-480c-9037-9a3d61dc5463 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:06:17.887116 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb13c529-1836-480c-9037-9a3d61dc5463 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:06:17.890094 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-282e00ae-b936-44f1-8c7e-24527237e9a2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0602cf79916d42c0ba0e7bd303b5e45d'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:06:17.918286 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1073/2160] 10.222.0.40 () {60 vars in 1144 bytes} [Fri Aug 26 11:06:17 2022] POST /identity/v3/users => generated 291 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 11:06:17.918286 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb13c529-1836-480c-9037-9a3d61dc5463 admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:06:17.918524 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1088/2161] 10.222.0.40 () {58 vars in 1224 bytes} [Fri Aug 26 11:06:17 2022] DELETE /identity/v3/users/0602cf79916d42c0ba0e7bd303b5e45d => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 11:06:17.919399 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282e00ae-b936-44f1-8c7e-24527237e9a2 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:06:17.922116 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-cb13c529-1836-480c-9037-9a3d61dc5463 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:06:17.923071 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-282e00ae-b936-44f1-8c7e-24527237e9a2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:06:17.935390 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb13c529-1836-480c-9037-9a3d61dc5463 admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:06:17.976612 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=5 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 11:06:18.034022 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282e00ae-b936-44f1-8c7e-24527237e9a2 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:06:18.037737 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1036d48-3e9e-417f-95ae-ca4889d4d200 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:06:18.038938 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1036d48-3e9e-417f-95ae-ca4889d4d200 admin admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:06:18.039224 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1036d48-3e9e-417f-95ae-ca4889d4d200 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:06:18.039498 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1036d48-3e9e-417f-95ae-ca4889d4d200 admin admin] PATH_INFO: `/v3/roles` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:06:18.040161 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1036d48-3e9e-417f-95ae-ca4889d4d200 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:06:18.042743 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1036d48-3e9e-417f-95ae-ca4889d4d200 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:06:18.053722 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 11:06:18.056184 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f824088-415d-4c3b-846a-d38b51c23093 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:06:18.057755 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f824088-415d-4c3b-846a-d38b51c23093 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:06:18.058364 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f824088-415d-4c3b-846a-d38b51c23093 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:06:18.058874 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f824088-415d-4c3b-846a-d38b51c23093 admin admin] PATH_INFO: `/v3/projects/57a6b40b9bcb4840851fb48cb9a19f6d` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:06:18.059272 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1089/2162] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 26 11:06:17 2022] GET /identity/v3/roles => generated 1420 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 11:06:18.063547 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f824088-415d-4c3b-846a-d38b51c23093 admin admin] RBAC: Authorizing `identity:delete_project(project_id=57a6b40b9bcb4840851fb48cb9a19f6d)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:06:18.064761 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1036d48-3e9e-417f-95ae-ca4889d4d200 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:06:18.065869 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f824088-415d-4c3b-846a-d38b51c23093 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:06:18.078270 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1036d48-3e9e-417f-95ae-ca4889d4d200 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:06:18.085682 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167ee5cc-5cee-4cf2-b412-60747a1e0fdf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:06:18.086443 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167ee5cc-5cee-4cf2-b412-60747a1e0fdf admin admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:06:18.086980 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167ee5cc-5cee-4cf2-b412-60747a1e0fdf admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:06:18.087318 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167ee5cc-5cee-4cf2-b412-60747a1e0fdf admin admin] PATH_INFO: `/v3/roles` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:06:18.088486 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-167ee5cc-5cee-4cf2-b412-60747a1e0fdf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:06:18.090170 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-167ee5cc-5cee-4cf2-b412-60747a1e0fdf admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:06:18.106032 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1090/2163] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 26 11:06:18 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 11:06:18.110791 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167ee5cc-5cee-4cf2-b412-60747a1e0fdf admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:06:18.123446 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167ee5cc-5cee-4cf2-b412-60747a1e0fdf admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:06:18.134087 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-7f824088-415d-4c3b-846a-d38b51c23093 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '57a6b40b9bcb4840851fb48cb9a19f6d', 'request_id': 'req-7f824088-415d-4c3b-846a-d38b51c23093'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:06:18.141513 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-7f824088-415d-4c3b-846a-d38b51c23093 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '57a6b40b9bcb4840851fb48cb9a19f6d', 'request_id': 'req-7f824088-415d-4c3b-846a-d38b51c23093'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:06:18.149269 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-7f824088-415d-4c3b-846a-d38b51c23093 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '57a6b40b9bcb4840851fb48cb9a19f6d', 'request_id': 'req-7f824088-415d-4c3b-846a-d38b51c23093'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:06:18.166526 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-7f824088-415d-4c3b-846a-d38b51c23093 admin admin] The token cache is being invalidate because project 57a6b40b9bcb4840851fb48cb9a19f6d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81423) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 11:06:18.166938 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-7f824088-415d-4c3b-846a-d38b51c23093 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:06:18.168070 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-7f824088-415d-4c3b-846a-d38b51c23093 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:06:18.170302 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1074/2164] 10.222.0.40 () {58 vars in 1233 bytes} [Fri Aug 26 11:06:17 2022] DELETE /identity/v3/projects/57a6b40b9bcb4840851fb48cb9a19f6d => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 11:06:18.174773 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2774b1f-805d-40da-a1d2-645fe00d3e11 None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:06:18.175278 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2774b1f-805d-40da-a1d2-645fe00d3e11 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:06:18.175569 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2774b1f-805d-40da-a1d2-645fe00d3e11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:06:18.185790 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13575c16-f91c-4c40-b877-f5ca9b141bb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:06:18.186611 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13575c16-f91c-4c40-b877-f5ca9b141bb3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:06:18.186986 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13575c16-f91c-4c40-b877-f5ca9b141bb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:06:18.187349 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13575c16-f91c-4c40-b877-f5ca9b141bb3 admin admin] PATH_INFO: `/v3/projects/76ea85839a1649d1aec671ca0d08acd8/users/80ea8098b42b4732944af30961e594f1/roles/23e174ce743b46eabe618c212691da55` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:06:18.198227 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-e2774b1f-805d-40da-a1d2-645fe00d3e11 None None] MFA Rules not processed for user `ca155a48e2024c32af3fd2e19e5fac11`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 11:06:18.199516 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-e2774b1f-805d-40da-a1d2-645fe00d3e11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:06:18.203468 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13575c16-f91c-4c40-b877-f5ca9b141bb3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=76ea85839a1649d1aec671ca0d08acd8, user_id=80ea8098b42b4732944af30961e594f1, role_id=23e174ce743b46eabe618c212691da55)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:06:18.209642 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13575c16-f91c-4c40-b877-f5ca9b141bb3 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:06:18.214056 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1075/2165] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 11:06:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 11:06:18.222615 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8800afe0-0470-48b6-84c0-9e1a16e2c3ca None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:06:18.223368 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8800afe0-0470-48b6-84c0-9e1a16e2c3ca None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:06:18.223368 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8800afe0-0470-48b6-84c0-9e1a16e2c3ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:06:18.230707 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1091/2166] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 26 11:06:18 2022] PUT /identity/v3/projects/76ea85839a1649d1aec671ca0d08acd8/users/80ea8098b42b4732944af30961e594f1/roles/23e174ce743b46eabe618c212691da55 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 11:06:18.241744 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f678ab08-5fe1-4ed6-a194-add31123ee2b None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:06:18.242098 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f678ab08-5fe1-4ed6-a194-add31123ee2b None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:06:18.242699 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f678ab08-5fe1-4ed6-a194-add31123ee2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:06:18.250463 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-8800afe0-0470-48b6-84c0-9e1a16e2c3ca None None] MFA Rules not processed for user `ca155a48e2024c32af3fd2e19e5fac11`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 11:06:18.252043 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-8800afe0-0470-48b6-84c0-9e1a16e2c3ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:06:18.273277 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-f678ab08-5fe1-4ed6-a194-add31123ee2b None None] MFA Rules not processed for user `80ea8098b42b4732944af30961e594f1`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 11:06:18.274563 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-f678ab08-5fe1-4ed6-a194-add31123ee2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:06:18.335620 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1076/2167] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 11:06:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 11:06:18.336558 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8800afe0-0470-48b6-84c0-9e1a16e2c3ca None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:06:18.337892 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-8800afe0-0470-48b6-84c0-9e1a16e2c3ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:06:18.350484 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8800afe0-0470-48b6-84c0-9e1a16e2c3ca None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:06:18.353646 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1092/2168] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 11:06:18 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 11:06:18.354509 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f678ab08-5fe1-4ed6-a194-add31123ee2b None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:06:18.355287 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f12b26c-5e9e-4492-ab50-1f1ce010aee7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:06:18.355910 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f12b26c-5e9e-4492-ab50-1f1ce010aee7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:06:18.356187 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f12b26c-5e9e-4492-ab50-1f1ce010aee7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:06:18.356578 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f12b26c-5e9e-4492-ab50-1f1ce010aee7 admin admin] PATH_INFO: `/v3/projects/9e232a2c10f0481ab44a03cacbd27845` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:06:18.357807 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f12b26c-5e9e-4492-ab50-1f1ce010aee7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9e232a2c10f0481ab44a03cacbd27845)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:06:18.359757 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f12b26c-5e9e-4492-ab50-1f1ce010aee7 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:06:18.368109 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f678ab08-5fe1-4ed6-a194-add31123ee2b None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:06:18.372811 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc4f3dc-1ee2-4cf6-a6e2-db36604edd63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:06:18.373398 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc4f3dc-1ee2-4cf6-a6e2-db36604edd63 admin admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:06:18.373670 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc4f3dc-1ee2-4cf6-a6e2-db36604edd63 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:06:18.373940 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc4f3dc-1ee2-4cf6-a6e2-db36604edd63 admin admin] PATH_INFO: `/v3/domains` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:06:18.374608 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc4f3dc-1ee2-4cf6-a6e2-db36604edd63 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 11:06:18.374904 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc4f3dc-1ee2-4cf6-a6e2-db36604edd63 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:06:18.376489 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc4f3dc-1ee2-4cf6-a6e2-db36604edd63 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:06:18.396044 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1093/2169] 10.222.0.40 () {58 vars in 1153 bytes} [Fri Aug 26 11:06:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 11:06:18.396044 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc4f3dc-1ee2-4cf6-a6e2-db36604edd63 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:06:18.396044 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-ecc4f3dc-1ee2-4cf6-a6e2-db36604edd63 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:06:18.405193 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 11:06:18.408558 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc4f3dc-1ee2-4cf6-a6e2-db36604edd63 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:06:18.413185 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-0f12b26c-5e9e-4492-ab50-1f1ce010aee7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9e232a2c10f0481ab44a03cacbd27845', 'request_id': 'req-0f12b26c-5e9e-4492-ab50-1f1ce010aee7'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:06:18.421446 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-0f12b26c-5e9e-4492-ab50-1f1ce010aee7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9e232a2c10f0481ab44a03cacbd27845', 'request_id': 'req-0f12b26c-5e9e-4492-ab50-1f1ce010aee7'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:06:18.427167 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-0f12b26c-5e9e-4492-ab50-1f1ce010aee7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9e232a2c10f0481ab44a03cacbd27845', 'request_id': 'req-0f12b26c-5e9e-4492-ab50-1f1ce010aee7'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:06:18.441852 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-0f12b26c-5e9e-4492-ab50-1f1ce010aee7 admin admin] The token cache is being invalidate because project 9e232a2c10f0481ab44a03cacbd27845 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81423) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 11:06:18.442174 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-0f12b26c-5e9e-4492-ab50-1f1ce010aee7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:06:18.442825 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-0f12b26c-5e9e-4492-ab50-1f1ce010aee7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:06:18.444493 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1077/2170] 10.222.0.40 () {58 vars in 1233 bytes} [Fri Aug 26 11:06:18 2022] DELETE /identity/v3/projects/9e232a2c10f0481ab44a03cacbd27845 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 11:06:18.445237 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f12b26c-5e9e-4492-ab50-1f1ce010aee7 admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:06:18.446543 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-0f12b26c-5e9e-4492-ab50-1f1ce010aee7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:06:18.464711 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 11:06:18.479476 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b707b762-7dd9-4b3f-9e44-c38f8515cddf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:06:18.480099 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b707b762-7dd9-4b3f-9e44-c38f8515cddf admin admin] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:06:18.480383 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b707b762-7dd9-4b3f-9e44-c38f8515cddf admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:06:18.480657 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b707b762-7dd9-4b3f-9e44-c38f8515cddf admin admin] PATH_INFO: `/v3/projects` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:06:18.481287 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b707b762-7dd9-4b3f-9e44-c38f8515cddf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:06:18.483231 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b707b762-7dd9-4b3f-9e44-c38f8515cddf admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:06:18.513543 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1094/2171] 10.222.0.40 () {60 vars in 1153 bytes} [Fri Aug 26 11:06:18 2022] POST /identity/v3/projects => generated 379 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 11:06:18.514731 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac39e349-7e7a-4baf-9927-ffce3e1938a6 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:06:18.515018 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac39e349-7e7a-4baf-9927-ffce3e1938a6 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:06:18.515313 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac39e349-7e7a-4baf-9927-ffce3e1938a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:06:18.518811 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f12b26c-5e9e-4492-ab50-1f1ce010aee7 admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:06:18.534900 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-ac39e349-7e7a-4baf-9927-ffce3e1938a6 None None] MFA Rules not processed for user `ca155a48e2024c32af3fd2e19e5fac11`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 11:06:18.536129 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-ac39e349-7e7a-4baf-9927-ffce3e1938a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:06:18.576827 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16086ef5-e36a-4b75-8176-8a5b97214e85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:06:18.577449 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16086ef5-e36a-4b75-8176-8a5b97214e85 admin admin] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:06:18.577727 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16086ef5-e36a-4b75-8176-8a5b97214e85 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:06:18.578026 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16086ef5-e36a-4b75-8176-8a5b97214e85 admin admin] PATH_INFO: `/v3/projects` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:06:18.578694 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16086ef5-e36a-4b75-8176-8a5b97214e85 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:06:18.580736 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16086ef5-e36a-4b75-8176-8a5b97214e85 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:06:18.607760 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1095/2172] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 11:06:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 11:06:18.607760 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac39e349-7e7a-4baf-9927-ffce3e1938a6 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:06:18.607760 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-ac39e349-7e7a-4baf-9927-ffce3e1938a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:06:18.608738 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1078/2173] 10.222.0.40 () {60 vars in 1153 bytes} [Fri Aug 26 11:06:18 2022] POST /identity/v3/projects => generated 385 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 11:06:18.610052 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b174c3c-58c2-4a96-9416-d9ca5bf864d8 None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:06:18.610657 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b174c3c-58c2-4a96-9416-d9ca5bf864d8 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:06:18.610933 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b174c3c-58c2-4a96-9416-d9ca5bf864d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:06:18.619879 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac39e349-7e7a-4baf-9927-ffce3e1938a6 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:06:18.632143 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-9b174c3c-58c2-4a96-9416-d9ca5bf864d8 None None] MFA Rules not processed for user `ca155a48e2024c32af3fd2e19e5fac11`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 11:06:18.633351 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-9b174c3c-58c2-4a96-9416-d9ca5bf864d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:06:18.682408 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e69adfd-7c54-4510-a12c-5bf8d6c32faa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:06:18.683020 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e69adfd-7c54-4510-a12c-5bf8d6c32faa admin admin] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:06:18.683300 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e69adfd-7c54-4510-a12c-5bf8d6c32faa admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:06:18.683857 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e69adfd-7c54-4510-a12c-5bf8d6c32faa admin admin] PATH_INFO: `/v3/users` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:06:18.684300 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e69adfd-7c54-4510-a12c-5bf8d6c32faa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:06:18.686232 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e69adfd-7c54-4510-a12c-5bf8d6c32faa admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:06:18.698904 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1079/2174] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 11:06:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 11:06:18.699773 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b174c3c-58c2-4a96-9416-d9ca5bf864d8 None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:06:18.704992 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=94,keystone:DELETE=4,keystone:INSERT=2 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 11:06:18.710185 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1096/2175] 10.222.0.40 () {60 vars in 1144 bytes} [Fri Aug 26 11:06:18 2022] POST /identity/v3/users => generated 296 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 11:06:18.711063 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e69adfd-7c54-4510-a12c-5bf8d6c32faa admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:06:18.711683 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b174c3c-58c2-4a96-9416-d9ca5bf864d8 None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:06:18.717898 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a9a755-1be3-43ff-a99b-c1bb7981a437 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:06:18.718349 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a9a755-1be3-43ff-a99b-c1bb7981a437 admin admin] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:06:18.718939 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a9a755-1be3-43ff-a99b-c1bb7981a437 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:06:18.720234 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a9a755-1be3-43ff-a99b-c1bb7981a437 admin admin] PATH_INFO: `/v3/users` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:06:18.720862 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a9a755-1be3-43ff-a99b-c1bb7981a437 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:06:18.725115 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a9a755-1be3-43ff-a99b-c1bb7981a437 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:06:18.725269 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e69adfd-7c54-4510-a12c-5bf8d6c32faa admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:06:18.730359 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ffa99a-73fe-4170-a9ba-62c8c1cd45c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:06:18.730997 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ffa99a-73fe-4170-a9ba-62c8c1cd45c3 admin admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:06:18.731549 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ffa99a-73fe-4170-a9ba-62c8c1cd45c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:06:18.731823 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ffa99a-73fe-4170-a9ba-62c8c1cd45c3 admin admin] PATH_INFO: `/v3/domains` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:06:18.732483 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ffa99a-73fe-4170-a9ba-62c8c1cd45c3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 11:06:18.732787 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ffa99a-73fe-4170-a9ba-62c8c1cd45c3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:06:18.734775 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ffa99a-73fe-4170-a9ba-62c8c1cd45c3 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:06:18.749931 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1080/2176] 10.222.0.40 () {60 vars in 1144 bytes} [Fri Aug 26 11:06:18 2022] POST /identity/v3/users => generated 293 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 11:06:18.749931 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a9a755-1be3-43ff-a99b-c1bb7981a437 admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:06:18.753374 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1097/2177] 10.222.0.40 () {58 vars in 1153 bytes} [Fri Aug 26 11:06:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 11:06:18.754386 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ffa99a-73fe-4170-a9ba-62c8c1cd45c3 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:06:18.758154 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a9a755-1be3-43ff-a99b-c1bb7981a437 admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:06:18.762970 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53fb8c35-5611-4f89-bf5a-9635cdf12246 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:06:18.763561 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fb8c35-5611-4f89-bf5a-9635cdf12246 admin admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:06:18.763842 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fb8c35-5611-4f89-bf5a-9635cdf12246 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:06:18.764112 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fb8c35-5611-4f89-bf5a-9635cdf12246 admin admin] PATH_INFO: `/v3/roles` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:06:18.764751 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53fb8c35-5611-4f89-bf5a-9635cdf12246 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:06:18.766041 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ffa99a-73fe-4170-a9ba-62c8c1cd45c3 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:06:18.766370 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53fb8c35-5611-4f89-bf5a-9635cdf12246 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:06:18.770763 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dfcca10-cc81-4023-8fd4-f95dac4e6ef4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:06:18.771391 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfcca10-cc81-4023-8fd4-f95dac4e6ef4 admin admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:06:18.771672 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfcca10-cc81-4023-8fd4-f95dac4e6ef4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:06:18.771940 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfcca10-cc81-4023-8fd4-f95dac4e6ef4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:06:18.772562 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dfcca10-cc81-4023-8fd4-f95dac4e6ef4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:06:18.774162 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dfcca10-cc81-4023-8fd4-f95dac4e6ef4 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:06:18.777066 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1081/2178] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 26 11:06:18 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 11:06:18.778519 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53fb8c35-5611-4f89-bf5a-9635cdf12246 admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:06:18.784262 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1098/2179] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 26 11:06:18 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 11:06:18.787942 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dfcca10-cc81-4023-8fd4-f95dac4e6ef4 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:06:18.788286 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53fb8c35-5611-4f89-bf5a-9635cdf12246 admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:06:18.793243 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff4c673-4b55-4eb4-a6e9-a06c8748d3e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:06:18.793958 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff4c673-4b55-4eb4-a6e9-a06c8748d3e5 admin admin] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:06:18.794278 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff4c673-4b55-4eb4-a6e9-a06c8748d3e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:06:18.794551 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff4c673-4b55-4eb4-a6e9-a06c8748d3e5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:06:18.795173 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff4c673-4b55-4eb4-a6e9-a06c8748d3e5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:06:18.797251 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff4c673-4b55-4eb4-a6e9-a06c8748d3e5 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:06:18.797625 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dfcca10-cc81-4023-8fd4-f95dac4e6ef4 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:06:18.802863 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ad1c0e-e88b-425c-a289-028d2b39198b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:06:18.803719 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ad1c0e-e88b-425c-a289-028d2b39198b admin admin] REQUEST_METHOD: `PUT` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:06:18.803994 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ad1c0e-e88b-425c-a289-028d2b39198b admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:06:18.804259 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ad1c0e-e88b-425c-a289-028d2b39198b admin admin] PATH_INFO: `/v3/projects/eeee6cbac36f46b7b04eb2e64ccf9745/users/fb314c7c741d46b4a698a9856ebd9e7b/roles/1f97e4602e2e4d6498913a81f9ae8d6a` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:06:18.820143 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ad1c0e-e88b-425c-a289-028d2b39198b admin admin] RBAC: Authorizing `identity:create_grant(project_id=eeee6cbac36f46b7b04eb2e64ccf9745, user_id=fb314c7c741d46b4a698a9856ebd9e7b, role_id=1f97e4602e2e4d6498913a81f9ae8d6a)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:06:18.822907 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ad1c0e-e88b-425c-a289-028d2b39198b admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:06:18.831436 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1082/2180] 10.222.0.40 () {60 vars in 1153 bytes} [Fri Aug 26 11:06:18 2022] POST /identity/v3/projects => generated 403 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 11:06:18.832433 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff4c673-4b55-4eb4-a6e9-a06c8748d3e5 admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:06:18.840454 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1099/2181] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 26 11:06:18 2022] PUT /identity/v3/projects/eeee6cbac36f46b7b04eb2e64ccf9745/users/fb314c7c741d46b4a698a9856ebd9e7b/roles/1f97e4602e2e4d6498913a81f9ae8d6a => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 11:06:18.841744 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ad1c0e-e88b-425c-a289-028d2b39198b admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:06:18.861421 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 11:06:18.905859 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff4c673-4b55-4eb4-a6e9-a06c8748d3e5 admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:06:18.908704 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ad1c0e-e88b-425c-a289-028d2b39198b admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:06:18.914212 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae7bd2b-e6c9-475f-b6eb-ac9962685eb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:06:18.914883 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae7bd2b-e6c9-475f-b6eb-ac9962685eb1 admin admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:06:18.915165 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae7bd2b-e6c9-475f-b6eb-ac9962685eb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:06:18.915434 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae7bd2b-e6c9-475f-b6eb-ac9962685eb1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:06:18.916081 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae7bd2b-e6c9-475f-b6eb-ac9962685eb1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:06:18.916280 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5da091-ae3a-48d5-bbda-c84596adf054 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:06:18.916974 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5da091-ae3a-48d5-bbda-c84596adf054 admin admin] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:06:18.917211 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5da091-ae3a-48d5-bbda-c84596adf054 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:06:18.917461 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5da091-ae3a-48d5-bbda-c84596adf054 admin admin] PATH_INFO: `/v3/users` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:06:18.917718 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae7bd2b-e6c9-475f-b6eb-ac9962685eb1 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:06:18.918130 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea5da091-ae3a-48d5-bbda-c84596adf054 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:06:18.920249 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea5da091-ae3a-48d5-bbda-c84596adf054 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:06:18.928793 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1083/2182] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 26 11:06:18 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 11:06:18.929626 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae7bd2b-e6c9-475f-b6eb-ac9962685eb1 admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:06:18.939893 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae7bd2b-e6c9-475f-b6eb-ac9962685eb1 admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:06:18.944117 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1100/2183] 10.222.0.40 () {60 vars in 1144 bytes} [Fri Aug 26 11:06:18 2022] POST /identity/v3/users => generated 302 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 11:06:18.946213 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5da091-ae3a-48d5-bbda-c84596adf054 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:06:18.947823 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5d2d30-76d2-4e5a-a383-784663a31d5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:06:18.948424 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5d2d30-76d2-4e5a-a383-784663a31d5e admin admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:06:18.948697 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5d2d30-76d2-4e5a-a383-784663a31d5e admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:06:18.948985 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5d2d30-76d2-4e5a-a383-784663a31d5e admin admin] PATH_INFO: `/v3/roles` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:06:18.949646 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a5d2d30-76d2-4e5a-a383-784663a31d5e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:06:18.951253 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a5d2d30-76d2-4e5a-a383-784663a31d5e admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:06:18.957128 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5da091-ae3a-48d5-bbda-c84596adf054 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:06:18.962625 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1084/2184] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 26 11:06:18 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 11:06:18.962788 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee046a8-1119-4448-b92e-37c0afd65956 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:06:18.963439 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5d2d30-76d2-4e5a-a383-784663a31d5e admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:06:18.963553 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee046a8-1119-4448-b92e-37c0afd65956 admin admin] REQUEST_METHOD: `PUT` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:06:18.963806 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee046a8-1119-4448-b92e-37c0afd65956 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:06:18.964082 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee046a8-1119-4448-b92e-37c0afd65956 admin admin] PATH_INFO: `/v3/projects/437b6fe232f14d8e99a03d1cae243561/users/81d53661005f47e8a326dd701ca93fb3/roles/23e174ce743b46eabe618c212691da55` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:06:18.974910 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5d2d30-76d2-4e5a-a383-784663a31d5e admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:06:18.979364 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a042b13a-c47e-4d94-946b-0cb3bb8a0d13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:06:18.979974 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a042b13a-c47e-4d94-946b-0cb3bb8a0d13 admin admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:06:18.980216 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a042b13a-c47e-4d94-946b-0cb3bb8a0d13 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:06:18.980488 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a042b13a-c47e-4d94-946b-0cb3bb8a0d13 admin admin] PATH_INFO: `/v3/roles` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:06:18.981417 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a042b13a-c47e-4d94-946b-0cb3bb8a0d13 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:06:18.983031 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a042b13a-c47e-4d94-946b-0cb3bb8a0d13 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:06:18.986646 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee046a8-1119-4448-b92e-37c0afd65956 admin admin] RBAC: Authorizing `identity:create_grant(project_id=437b6fe232f14d8e99a03d1cae243561, user_id=81d53661005f47e8a326dd701ca93fb3, role_id=23e174ce743b46eabe618c212691da55)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:06:18.989436 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee046a8-1119-4448-b92e-37c0afd65956 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:06:18.999252 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1085/2185] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 26 11:06:18 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 11:06:18.999252 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a042b13a-c47e-4d94-946b-0cb3bb8a0d13 admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:06:19.008714 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a042b13a-c47e-4d94-946b-0cb3bb8a0d13 admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:06:19.009915 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1101/2186] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 26 11:06:18 2022] PUT /identity/v3/projects/437b6fe232f14d8e99a03d1cae243561/users/81d53661005f47e8a326dd701ca93fb3/roles/23e174ce743b46eabe618c212691da55 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 11:06:19.011059 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee046a8-1119-4448-b92e-37c0afd65956 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:06:19.070575 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68684292-4c8c-4a07-b353-d36cca664227 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:06:19.071231 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68684292-4c8c-4a07-b353-d36cca664227 admin admin] REQUEST_METHOD: `PUT` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:06:19.071511 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68684292-4c8c-4a07-b353-d36cca664227 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:06:19.071779 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68684292-4c8c-4a07-b353-d36cca664227 admin admin] PATH_INFO: `/v3/projects/eeee6cbac36f46b7b04eb2e64ccf9745/users/fb314c7c741d46b4a698a9856ebd9e7b/roles/1f97e4602e2e4d6498913a81f9ae8d6a` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:06:19.073845 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68684292-4c8c-4a07-b353-d36cca664227 admin admin] RBAC: Authorizing `identity:create_grant(project_id=eeee6cbac36f46b7b04eb2e64ccf9745, user_id=fb314c7c741d46b4a698a9856ebd9e7b, role_id=1f97e4602e2e4d6498913a81f9ae8d6a)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:06:19.076616 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68684292-4c8c-4a07-b353-d36cca664227 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:06:19.092344 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1086/2187] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 26 11:06:18 2022] PUT /identity/v3/projects/eeee6cbac36f46b7b04eb2e64ccf9745/users/fb314c7c741d46b4a698a9856ebd9e7b/roles/1f97e4602e2e4d6498913a81f9ae8d6a => generated 0 bytes in 95 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 11:06:19.093480 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad7e6db-7a96-4bbd-84a3-f28d11b819da None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:06:19.093771 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad7e6db-7a96-4bbd-84a3-f28d11b819da None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:06:19.094078 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad7e6db-7a96-4bbd-84a3-f28d11b819da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:06:19.112430 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee046a8-1119-4448-b92e-37c0afd65956 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:06:19.124645 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-fad7e6db-7a96-4bbd-84a3-f28d11b819da None None] MFA Rules not processed for user `81d53661005f47e8a326dd701ca93fb3`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 11:06:19.126083 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-fad7e6db-7a96-4bbd-84a3-f28d11b819da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:06:19.138320 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 11:06:19.180414 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003125bc-49c1-4537-9e85-a5fb414ea6ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:06:19.181741 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003125bc-49c1-4537-9e85-a5fb414ea6ff admin admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:06:19.181741 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003125bc-49c1-4537-9e85-a5fb414ea6ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:06:19.181741 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003125bc-49c1-4537-9e85-a5fb414ea6ff admin admin] PATH_INFO: `/v3/roles` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:06:19.182495 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-003125bc-49c1-4537-9e85-a5fb414ea6ff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:06:19.184125 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-003125bc-49c1-4537-9e85-a5fb414ea6ff admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:06:19.197684 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1102/2188] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 26 11:06:19 2022] GET /identity/v3/roles => generated 1420 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 11:06:19.197684 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c38d336-fe07-4e0e-bc95-423471be567a None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:06:19.197684 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c38d336-fe07-4e0e-bc95-423471be567a None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:06:19.197684 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c38d336-fe07-4e0e-bc95-423471be567a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:06:19.207846 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1087/2189] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 11:06:19 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 11:06:19.211647 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad7e6db-7a96-4bbd-84a3-f28d11b819da None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:06:19.227680 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad7e6db-7a96-4bbd-84a3-f28d11b819da None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:06:19.229795 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-7c38d336-fe07-4e0e-bc95-423471be567a None None] MFA Rules not processed for user `fb314c7c741d46b4a698a9856ebd9e7b`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 11:06:19.231325 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-7c38d336-fe07-4e0e-bc95-423471be567a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:06:19.233581 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19da39c-826b-4027-bfa0-041b59e02685 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:06:19.234544 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19da39c-826b-4027-bfa0-041b59e02685 admin admin] REQUEST_METHOD: `PUT` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:06:19.234683 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19da39c-826b-4027-bfa0-041b59e02685 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:06:19.234859 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19da39c-826b-4027-bfa0-041b59e02685 admin admin] PATH_INFO: `/v3/projects/99e7045fd2644e19b829b93e494b3a51/users/1ca9c5f32486425daec1f85a381e95cc/roles/23e174ce743b46eabe618c212691da55` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:06:19.249144 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 11:06:19.254639 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d19da39c-826b-4027-bfa0-041b59e02685 admin admin] RBAC: Authorizing `identity:create_grant(project_id=99e7045fd2644e19b829b93e494b3a51, user_id=1ca9c5f32486425daec1f85a381e95cc, role_id=23e174ce743b46eabe618c212691da55)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:06:19.257844 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d19da39c-826b-4027-bfa0-041b59e02685 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:06:19.275572 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1088/2190] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 26 11:06:19 2022] PUT /identity/v3/projects/99e7045fd2644e19b829b93e494b3a51/users/1ca9c5f32486425daec1f85a381e95cc/roles/23e174ce743b46eabe618c212691da55 => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 11:06:19.276480 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19da39c-826b-4027-bfa0-041b59e02685 admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:06:19.278160 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-d19da39c-826b-4027-bfa0-041b59e02685 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:06:19.334112 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1103/2191] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 11:06:19 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 11:06:19.335822 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e7b8fe-721c-45d2-91ab-c7ddf06f9692 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:06:19.336294 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e7b8fe-721c-45d2-91ab-c7ddf06f9692 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:06:19.336728 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e7b8fe-721c-45d2-91ab-c7ddf06f9692 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:06:19.342810 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19da39c-826b-4027-bfa0-041b59e02685 admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:06:19.346821 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5b9787-f211-480f-b7ad-3a207371af9c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c277338a7f814cc6b01c5419775031bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e30ccf0c1204727b786876c141dbcd2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:06:19.347422 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5b9787-f211-480f-b7ad-3a207371af9c service cinder] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:06:19.347698 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5b9787-f211-480f-b7ad-3a207371af9c service cinder] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:06:19.347987 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5b9787-f211-480f-b7ad-3a207371af9c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:06:19.358386 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-59e7b8fe-721c-45d2-91ab-c7ddf06f9692 None None] MFA Rules not processed for user `ca155a48e2024c32af3fd2e19e5fac11`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 11:06:19.359615 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-59e7b8fe-721c-45d2-91ab-c7ddf06f9692 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:06:19.416818 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e5b9787-f211-480f-b7ad-3a207371af9c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:06:19.418320 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e5b9787-f211-480f-b7ad-3a207371af9c service cinder] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:06:19.426675 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1104/2192] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 11:06:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 11:06:19.428405 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e7b8fe-721c-45d2-91ab-c7ddf06f9692 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:06:19.428532 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1089/2193] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 11:06:19 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 11:06:19.431268 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-59e7b8fe-721c-45d2-91ab-c7ddf06f9692 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:06:19.438107 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5b9787-f211-480f-b7ad-3a207371af9c service cinder] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:06:19.440407 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-2e5b9787-f211-480f-b7ad-3a207371af9c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:06:19.444722 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e7b8fe-721c-45d2-91ab-c7ddf06f9692 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:06:19.448339 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868e899e-77e7-42ea-9d93-056c209a39b0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c277338a7f814cc6b01c5419775031bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e30ccf0c1204727b786876c141dbcd2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:06:19.448948 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868e899e-77e7-42ea-9d93-056c209a39b0 service cinder] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:06:19.449225 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868e899e-77e7-42ea-9d93-056c209a39b0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:06:19.449496 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868e899e-77e7-42ea-9d93-056c209a39b0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:06:19.496339 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 11:06:19.515252 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5b9787-f211-480f-b7ad-3a207371af9c service cinder] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:06:19.519979 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93721cec-3cf8-4a13-9a6b-0983a720fc30 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91da2acd2534423091537e1ba068d2b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e30ccf0c1204727b786876c141dbcd2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:06:19.520521 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-868e899e-77e7-42ea-9d93-056c209a39b0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:06:19.520955 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93721cec-3cf8-4a13-9a6b-0983a720fc30 service neutron] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:06:19.521960 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93721cec-3cf8-4a13-9a6b-0983a720fc30 service neutron] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:06:19.522145 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-868e899e-77e7-42ea-9d93-056c209a39b0 service cinder] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:06:19.522554 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93721cec-3cf8-4a13-9a6b-0983a720fc30 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:06:19.532632 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93721cec-3cf8-4a13-9a6b-0983a720fc30 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:06:19.533521 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1105/2194] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 11:06:19 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 11:06:19.534076 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93721cec-3cf8-4a13-9a6b-0983a720fc30 service neutron] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:06:19.544165 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1090/2195] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 11:06:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 26 11:06:19.957419 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868e899e-77e7-42ea-9d93-056c209a39b0 service cinder] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:06:19.969994 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868e899e-77e7-42ea-9d93-056c209a39b0 service cinder] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:06:19.976455 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e83db37-1aee-43b7-a68c-1e6337e8e0a8 tempest-VolumeQuotaClassesTest-374086104 tempest-VolumeQuotaClassesTest-374086104-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb314c7c741d46b4a698a9856ebd9e7b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee6cbac36f46b7b04eb2e64ccf9745', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:06:19.977203 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e83db37-1aee-43b7-a68c-1e6337e8e0a8 tempest-VolumeQuotaClassesTest-374086104 tempest-VolumeQuotaClassesTest-374086104-project-admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:06:19.977558 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e83db37-1aee-43b7-a68c-1e6337e8e0a8 tempest-VolumeQuotaClassesTest-374086104 tempest-VolumeQuotaClassesTest-374086104-project-admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:06:19.977892 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e83db37-1aee-43b7-a68c-1e6337e8e0a8 tempest-VolumeQuotaClassesTest-374086104 tempest-VolumeQuotaClassesTest-374086104-project-admin] PATH_INFO: `/v3/domains` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:06:19.978688 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e83db37-1aee-43b7-a68c-1e6337e8e0a8 tempest-VolumeQuotaClassesTest-374086104 tempest-VolumeQuotaClassesTest-374086104-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 11:06:19.979040 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e83db37-1aee-43b7-a68c-1e6337e8e0a8 tempest-VolumeQuotaClassesTest-374086104 tempest-VolumeQuotaClassesTest-374086104-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:06:19.980757 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e83db37-1aee-43b7-a68c-1e6337e8e0a8 tempest-VolumeQuotaClassesTest-374086104 tempest-VolumeQuotaClassesTest-374086104-project-admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:06:19.997133 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1106/2196] 10.222.0.40 () {58 vars in 1153 bytes} [Fri Aug 26 11:06:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 11:06:20.001261 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93721cec-3cf8-4a13-9a6b-0983a720fc30 service neutron] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:06:20.023723 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93721cec-3cf8-4a13-9a6b-0983a720fc30 service neutron] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:06:20.030709 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30295eb-65eb-4a04-8e45-bed43cb4dd9d tempest-VolumeQuotaClassesTest-374086104 tempest-VolumeQuotaClassesTest-374086104-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb314c7c741d46b4a698a9856ebd9e7b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee6cbac36f46b7b04eb2e64ccf9745', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:06:20.031879 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30295eb-65eb-4a04-8e45-bed43cb4dd9d tempest-VolumeQuotaClassesTest-374086104 tempest-VolumeQuotaClassesTest-374086104-project-admin] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:06:20.032352 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30295eb-65eb-4a04-8e45-bed43cb4dd9d tempest-VolumeQuotaClassesTest-374086104 tempest-VolumeQuotaClassesTest-374086104-project-admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:06:20.032837 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30295eb-65eb-4a04-8e45-bed43cb4dd9d tempest-VolumeQuotaClassesTest-374086104 tempest-VolumeQuotaClassesTest-374086104-project-admin] PATH_INFO: `/v3/projects` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:06:20.034175 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c30295eb-65eb-4a04-8e45-bed43cb4dd9d tempest-VolumeQuotaClassesTest-374086104 tempest-VolumeQuotaClassesTest-374086104-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:06:20.038250 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c30295eb-65eb-4a04-8e45-bed43cb4dd9d tempest-VolumeQuotaClassesTest-374086104 tempest-VolumeQuotaClassesTest-374086104-project-admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:06:20.080901 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1091/2197] 10.222.0.40 () {60 vars in 1153 bytes} [Fri Aug 26 11:06:20 2022] POST /identity/v3/projects => generated 354 bytes in 77 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 11:06:20.081852 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e83db37-1aee-43b7-a68c-1e6337e8e0a8 tempest-VolumeQuotaClassesTest-374086104 tempest-VolumeQuotaClassesTest-374086104-project-admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:06:20.096991 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e83db37-1aee-43b7-a68c-1e6337e8e0a8 tempest-VolumeQuotaClassesTest-374086104 tempest-VolumeQuotaClassesTest-374086104-project-admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:06:20.134484 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 11:06:20.173122 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f98582c-8f5c-4d08-a13c-d1e6ef2aaf86 tempest-VolumeQuotaClassesTest-374086104 tempest-VolumeQuotaClassesTest-374086104-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb314c7c741d46b4a698a9856ebd9e7b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee6cbac36f46b7b04eb2e64ccf9745', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:06:20.173738 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f98582c-8f5c-4d08-a13c-d1e6ef2aaf86 tempest-VolumeQuotaClassesTest-374086104 tempest-VolumeQuotaClassesTest-374086104-project-admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:06:20.174115 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f98582c-8f5c-4d08-a13c-d1e6ef2aaf86 tempest-VolumeQuotaClassesTest-374086104 tempest-VolumeQuotaClassesTest-374086104-project-admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:06:20.174409 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f98582c-8f5c-4d08-a13c-d1e6ef2aaf86 tempest-VolumeQuotaClassesTest-374086104 tempest-VolumeQuotaClassesTest-374086104-project-admin] PATH_INFO: `/v3/domains` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:06:20.175066 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f98582c-8f5c-4d08-a13c-d1e6ef2aaf86 tempest-VolumeQuotaClassesTest-374086104 tempest-VolumeQuotaClassesTest-374086104-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 11:06:20.175373 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f98582c-8f5c-4d08-a13c-d1e6ef2aaf86 tempest-VolumeQuotaClassesTest-374086104 tempest-VolumeQuotaClassesTest-374086104-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:06:20.177002 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f98582c-8f5c-4d08-a13c-d1e6ef2aaf86 tempest-VolumeQuotaClassesTest-374086104 tempest-VolumeQuotaClassesTest-374086104-project-admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:06:20.194828 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1107/2198] 10.222.0.40 () {58 vars in 1153 bytes} [Fri Aug 26 11:06:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 114 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 11:06:20.234762 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30295eb-65eb-4a04-8e45-bed43cb4dd9d tempest-VolumeQuotaClassesTest-374086104 tempest-VolumeQuotaClassesTest-374086104-project-admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:06:20.246186 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30295eb-65eb-4a04-8e45-bed43cb4dd9d tempest-VolumeQuotaClassesTest-374086104 tempest-VolumeQuotaClassesTest-374086104-project-admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:06:20.250951 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44604e79-5fb3-4108-a7b1-db96a3dedbd0 tempest-VolumeQuotaClassesTest-374086104 tempest-VolumeQuotaClassesTest-374086104-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb314c7c741d46b4a698a9856ebd9e7b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee6cbac36f46b7b04eb2e64ccf9745', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:06:20.251585 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44604e79-5fb3-4108-a7b1-db96a3dedbd0 tempest-VolumeQuotaClassesTest-374086104 tempest-VolumeQuotaClassesTest-374086104-project-admin] REQUEST_METHOD: `DELETE` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:06:20.251854 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44604e79-5fb3-4108-a7b1-db96a3dedbd0 tempest-VolumeQuotaClassesTest-374086104 tempest-VolumeQuotaClassesTest-374086104-project-admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:06:20.252126 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44604e79-5fb3-4108-a7b1-db96a3dedbd0 tempest-VolumeQuotaClassesTest-374086104 tempest-VolumeQuotaClassesTest-374086104-project-admin] PATH_INFO: `/v3/projects/531340da8afc48568806261fdcdea0aa` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:06:20.253256 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44604e79-5fb3-4108-a7b1-db96a3dedbd0 tempest-VolumeQuotaClassesTest-374086104 tempest-VolumeQuotaClassesTest-374086104-project-admin] RBAC: Authorizing `identity:delete_project(project_id=531340da8afc48568806261fdcdea0aa)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:06:20.255216 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44604e79-5fb3-4108-a7b1-db96a3dedbd0 tempest-VolumeQuotaClassesTest-374086104 tempest-VolumeQuotaClassesTest-374086104-project-admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:06:20.292916 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016a9a5f-5e07-455f-ab10-8ed007c0ca63 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:06:20.293212 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016a9a5f-5e07-455f-ab10-8ed007c0ca63 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:06:20.293543 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016a9a5f-5e07-455f-ab10-8ed007c0ca63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:06:20.338529 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-016a9a5f-5e07-455f-ab10-8ed007c0ca63 None None] MFA Rules not processed for user `1ca9c5f32486425daec1f85a381e95cc`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 11:06:20.342962 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-016a9a5f-5e07-455f-ab10-8ed007c0ca63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:06:20.343215 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-44604e79-5fb3-4108-a7b1-db96a3dedbd0 tempest-VolumeQuotaClassesTest-374086104 tempest-VolumeQuotaClassesTest-374086104-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '531340da8afc48568806261fdcdea0aa', 'request_id': 'req-44604e79-5fb3-4108-a7b1-db96a3dedbd0'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:06:20.353230 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-44604e79-5fb3-4108-a7b1-db96a3dedbd0 tempest-VolumeQuotaClassesTest-374086104 tempest-VolumeQuotaClassesTest-374086104-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '531340da8afc48568806261fdcdea0aa', 'request_id': 'req-44604e79-5fb3-4108-a7b1-db96a3dedbd0'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:06:20.362457 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-44604e79-5fb3-4108-a7b1-db96a3dedbd0 tempest-VolumeQuotaClassesTest-374086104 tempest-VolumeQuotaClassesTest-374086104-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '531340da8afc48568806261fdcdea0aa', 'request_id': 'req-44604e79-5fb3-4108-a7b1-db96a3dedbd0'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:06:20.381460 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-44604e79-5fb3-4108-a7b1-db96a3dedbd0 tempest-VolumeQuotaClassesTest-374086104 tempest-VolumeQuotaClassesTest-374086104-project-admin] The token cache is being invalidate because project 531340da8afc48568806261fdcdea0aa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81423) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 11:06:20.382046 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-44604e79-5fb3-4108-a7b1-db96a3dedbd0 tempest-VolumeQuotaClassesTest-374086104 tempest-VolumeQuotaClassesTest-374086104-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:06:20.383228 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-44604e79-5fb3-4108-a7b1-db96a3dedbd0 tempest-VolumeQuotaClassesTest-374086104 tempest-VolumeQuotaClassesTest-374086104-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:06:20.385634 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1092/2199] 10.222.0.40 () {58 vars in 1233 bytes} [Fri Aug 26 11:06:20 2022] DELETE /identity/v3/projects/531340da8afc48568806261fdcdea0aa => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 11:06:20.444468 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1108/2200] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 11:06:20 2022] POST /identity/v3/auth/tokens => generated 3280 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 11:06:20.448422 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44604e79-5fb3-4108-a7b1-db96a3dedbd0 tempest-VolumeQuotaClassesTest-374086104 tempest-VolumeQuotaClassesTest-374086104-project-admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:06:20.449914 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-44604e79-5fb3-4108-a7b1-db96a3dedbd0 tempest-VolumeQuotaClassesTest-374086104 tempest-VolumeQuotaClassesTest-374086104-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:06:20.543797 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44604e79-5fb3-4108-a7b1-db96a3dedbd0 tempest-VolumeQuotaClassesTest-374086104 tempest-VolumeQuotaClassesTest-374086104-project-admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:06:20.548447 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6d23e4-d210-4518-8fcd-504fc843bfb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:06:20.549099 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6d23e4-d210-4518-8fcd-504fc843bfb4 admin admin] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:06:20.549399 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6d23e4-d210-4518-8fcd-504fc843bfb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:06:20.549647 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6d23e4-d210-4518-8fcd-504fc843bfb4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:06:20.550344 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da6d23e4-d210-4518-8fcd-504fc843bfb4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:06:20.553056 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da6d23e4-d210-4518-8fcd-504fc843bfb4 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:06:20.581226 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1093/2201] 10.222.0.40 () {60 vars in 1153 bytes} [Fri Aug 26 11:06:20 2022] POST /identity/v3/projects => generated 405 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 11:06:20.585561 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016a9a5f-5e07-455f-ab10-8ed007c0ca63 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:06:20.596393 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016a9a5f-5e07-455f-ab10-8ed007c0ca63 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:06:20.636919 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6d23e4-d210-4518-8fcd-504fc843bfb4 admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:06:20.638775 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-da6d23e4-d210-4518-8fcd-504fc843bfb4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:06:20.662120 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5e670b-96c5-4773-9454-f362b266300d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:06:20.662827 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5e670b-96c5-4773-9454-f362b266300d admin admin] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:06:20.663099 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5e670b-96c5-4773-9454-f362b266300d admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:06:20.663367 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5e670b-96c5-4773-9454-f362b266300d admin admin] PATH_INFO: `/v3/users` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:06:20.664044 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f5e670b-96c5-4773-9454-f362b266300d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:06:20.665966 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f5e670b-96c5-4773-9454-f362b266300d admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:06:20.672068 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 11:06:20.689249 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1109/2202] 10.222.0.40 () {60 vars in 1144 bytes} [Fri Aug 26 11:06:20 2022] POST /identity/v3/users => generated 309 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 11:06:20.693728 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5e670b-96c5-4773-9454-f362b266300d admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:06:20.704075 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5e670b-96c5-4773-9454-f362b266300d admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:06:20.709511 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af20362d-7012-4dd1-842c-833ee193d1a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:06:20.710338 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af20362d-7012-4dd1-842c-833ee193d1a0 admin admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:06:20.710615 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af20362d-7012-4dd1-842c-833ee193d1a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:06:20.710881 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af20362d-7012-4dd1-842c-833ee193d1a0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:06:20.711517 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af20362d-7012-4dd1-842c-833ee193d1a0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:06:20.713095 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af20362d-7012-4dd1-842c-833ee193d1a0 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:06:20.716573 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6d23e4-d210-4518-8fcd-504fc843bfb4 admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:06:20.721705 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399ba2f1-8202-4ac0-8a5c-dc2db4bfbfc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:06:20.722369 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399ba2f1-8202-4ac0-8a5c-dc2db4bfbfc8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:06:20.722679 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399ba2f1-8202-4ac0-8a5c-dc2db4bfbfc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:06:20.722952 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399ba2f1-8202-4ac0-8a5c-dc2db4bfbfc8 admin admin] PATH_INFO: `/v3/users/80ea8098b42b4732944af30961e594f1` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:06:20.723590 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1110/2203] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 26 11:06:20 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 26 11:06:20.726607 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-399ba2f1-8202-4ac0-8a5c-dc2db4bfbfc8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=80ea8098b42b4732944af30961e594f1)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:06:20.727852 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af20362d-7012-4dd1-842c-833ee193d1a0 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:06:20.728806 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-399ba2f1-8202-4ac0-8a5c-dc2db4bfbfc8 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:06:20.737458 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af20362d-7012-4dd1-842c-833ee193d1a0 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:06:20.742151 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29557dad-51ed-4cbe-afdb-e812b38d385b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:06:20.743050 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29557dad-51ed-4cbe-afdb-e812b38d385b admin admin] REQUEST_METHOD: `PUT` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:06:20.743322 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29557dad-51ed-4cbe-afdb-e812b38d385b admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:06:20.743592 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29557dad-51ed-4cbe-afdb-e812b38d385b admin admin] PATH_INFO: `/v3/projects/80c6f1ee09494dbcae12bc8a557869c6/users/df77ae1fc34247bf975b1de8b3dd2e1e/roles/1f97e4602e2e4d6498913a81f9ae8d6a` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:06:20.759237 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29557dad-51ed-4cbe-afdb-e812b38d385b admin admin] RBAC: Authorizing `identity:create_grant(project_id=80c6f1ee09494dbcae12bc8a557869c6, user_id=df77ae1fc34247bf975b1de8b3dd2e1e, role_id=1f97e4602e2e4d6498913a81f9ae8d6a)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:06:20.762271 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29557dad-51ed-4cbe-afdb-e812b38d385b admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:06:20.776027 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1111/2204] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 26 11:06:20 2022] PUT /identity/v3/projects/80c6f1ee09494dbcae12bc8a557869c6/users/df77ae1fc34247bf975b1de8b3dd2e1e/roles/1f97e4602e2e4d6498913a81f9ae8d6a => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 26 11:06:20.780366 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29557dad-51ed-4cbe-afdb-e812b38d385b admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:06:20.783272 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-399ba2f1-8202-4ac0-8a5c-dc2db4bfbfc8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '80ea8098b42b4732944af30961e594f1'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:06:20.790856 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 11:06:20.792772 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-399ba2f1-8202-4ac0-8a5c-dc2db4bfbfc8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '80ea8098b42b4732944af30961e594f1'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:06:20.793502 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-399ba2f1-8202-4ac0-8a5c-dc2db4bfbfc8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '80ea8098b42b4732944af30961e594f1'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:06:20.806760 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-399ba2f1-8202-4ac0-8a5c-dc2db4bfbfc8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '80ea8098b42b4732944af30961e594f1'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:06:20.807451 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-399ba2f1-8202-4ac0-8a5c-dc2db4bfbfc8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '80ea8098b42b4732944af30961e594f1'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:06:20.821748 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1094/2205] 10.222.0.40 () {58 vars in 1224 bytes} [Fri Aug 26 11:06:20 2022] DELETE /identity/v3/users/80ea8098b42b4732944af30961e594f1 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 26 11:06:20.825892 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4d03de-c08c-47e3-b95a-f920d04653f5 None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:06:20.826206 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4d03de-c08c-47e3-b95a-f920d04653f5 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:06:20.826478 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4d03de-c08c-47e3-b95a-f920d04653f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:06:20.847066 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-3c4d03de-c08c-47e3-b95a-f920d04653f5 None None] MFA Rules not processed for user `ca155a48e2024c32af3fd2e19e5fac11`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 11:06:20.848300 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-3c4d03de-c08c-47e3-b95a-f920d04653f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:06:20.854662 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29557dad-51ed-4cbe-afdb-e812b38d385b admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:06:20.934367 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4f25fd-9c72-4997-b712-6eb5912e00ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:06:20.934987 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4f25fd-9c72-4997-b712-6eb5912e00ac admin admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:06:20.935268 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4f25fd-9c72-4997-b712-6eb5912e00ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:06:20.935537 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4f25fd-9c72-4997-b712-6eb5912e00ac admin admin] PATH_INFO: `/v3/roles` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:06:20.936175 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be4f25fd-9c72-4997-b712-6eb5912e00ac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:06:20.937910 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be4f25fd-9c72-4997-b712-6eb5912e00ac admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:06:20.941786 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1095/2206] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 11:06:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 11:06:20.952812 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4d03de-c08c-47e3-b95a-f920d04653f5 None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:06:20.952955 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1112/2207] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 26 11:06:20 2022] GET /identity/v3/roles => generated 1420 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 26 11:06:20.954651 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-3c4d03de-c08c-47e3-b95a-f920d04653f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:06:20.956781 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4f25fd-9c72-4997-b712-6eb5912e00ac admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:06:20.958616 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-be4f25fd-9c72-4997-b712-6eb5912e00ac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:06:20.972144 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4f25fd-9c72-4997-b712-6eb5912e00ac admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:06:20.991135 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346c6740-8f50-4a96-b82d-649143a9d68d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:06:20.991851 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346c6740-8f50-4a96-b82d-649143a9d68d admin admin] REQUEST_METHOD: `PUT` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:06:20.992128 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346c6740-8f50-4a96-b82d-649143a9d68d admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:06:20.992398 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346c6740-8f50-4a96-b82d-649143a9d68d admin admin] PATH_INFO: `/v3/projects/80c6f1ee09494dbcae12bc8a557869c6/users/df77ae1fc34247bf975b1de8b3dd2e1e/roles/1f97e4602e2e4d6498913a81f9ae8d6a` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:06:20.994634 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-346c6740-8f50-4a96-b82d-649143a9d68d admin admin] RBAC: Authorizing `identity:create_grant(project_id=80c6f1ee09494dbcae12bc8a557869c6, user_id=df77ae1fc34247bf975b1de8b3dd2e1e, role_id=1f97e4602e2e4d6498913a81f9ae8d6a)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:06:20.997950 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-346c6740-8f50-4a96-b82d-649143a9d68d admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:06:21.015054 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1113/2208] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 26 11:06:20 2022] PUT /identity/v3/projects/80c6f1ee09494dbcae12bc8a557869c6/users/df77ae1fc34247bf975b1de8b3dd2e1e/roles/1f97e4602e2e4d6498913a81f9ae8d6a => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 11:06:21.026216 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 11:06:21.030934 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4d03de-c08c-47e3-b95a-f920d04653f5 None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:06:21.105670 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9844e009-e967-4b50-b7a9-4c87657bbe6d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91da2acd2534423091537e1ba068d2b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e30ccf0c1204727b786876c141dbcd2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:06:21.107157 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9844e009-e967-4b50-b7a9-4c87657bbe6d service neutron] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:06:21.107486 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9844e009-e967-4b50-b7a9-4c87657bbe6d service neutron] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:06:21.107799 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9844e009-e967-4b50-b7a9-4c87657bbe6d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:06:21.189079 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9844e009-e967-4b50-b7a9-4c87657bbe6d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:06:21.190735 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9844e009-e967-4b50-b7a9-4c87657bbe6d service neutron] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:06:21.201843 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1096/2209] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 11:06:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 249 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 11:06:21.671572 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346c6740-8f50-4a96-b82d-649143a9d68d admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:06:21.673404 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-346c6740-8f50-4a96-b82d-649143a9d68d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:06:21.691072 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346c6740-8f50-4a96-b82d-649143a9d68d admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:06:21.706541 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6124ecb5-61c4-43d9-87d0-5a0391834b6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:06:21.707200 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6124ecb5-61c4-43d9-87d0-5a0391834b6d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:06:21.707493 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6124ecb5-61c4-43d9-87d0-5a0391834b6d admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:06:21.707852 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6124ecb5-61c4-43d9-87d0-5a0391834b6d admin admin] PATH_INFO: `/v3/projects/76ea85839a1649d1aec671ca0d08acd8` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:06:21.709897 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6124ecb5-61c4-43d9-87d0-5a0391834b6d admin admin] RBAC: Authorizing `identity:delete_project(project_id=76ea85839a1649d1aec671ca0d08acd8)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:06:21.712053 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6124ecb5-61c4-43d9-87d0-5a0391834b6d admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:06:21.764471 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-6124ecb5-61c4-43d9-87d0-5a0391834b6d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '76ea85839a1649d1aec671ca0d08acd8', 'request_id': 'req-6124ecb5-61c4-43d9-87d0-5a0391834b6d'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:06:21.774449 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-6124ecb5-61c4-43d9-87d0-5a0391834b6d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '76ea85839a1649d1aec671ca0d08acd8', 'request_id': 'req-6124ecb5-61c4-43d9-87d0-5a0391834b6d'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:06:21.781683 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-6124ecb5-61c4-43d9-87d0-5a0391834b6d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '76ea85839a1649d1aec671ca0d08acd8', 'request_id': 'req-6124ecb5-61c4-43d9-87d0-5a0391834b6d'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:06:21.801669 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-6124ecb5-61c4-43d9-87d0-5a0391834b6d admin admin] The token cache is being invalidate because project 76ea85839a1649d1aec671ca0d08acd8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81424) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 11:06:21.802875 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-6124ecb5-61c4-43d9-87d0-5a0391834b6d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:06:21.803943 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-6124ecb5-61c4-43d9-87d0-5a0391834b6d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:06:21.805661 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1114/2210] 10.222.0.40 () {58 vars in 1233 bytes} [Fri Aug 26 11:06:21 2022] DELETE /identity/v3/projects/76ea85839a1649d1aec671ca0d08acd8 => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 11:06:21.810186 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9844e009-e967-4b50-b7a9-4c87657bbe6d service neutron] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:06:21.821396 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9844e009-e967-4b50-b7a9-4c87657bbe6d service neutron] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:06:21.886629 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f109c2-aa3f-4a6a-939f-056566d9b804 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:06:21.887486 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f109c2-aa3f-4a6a-939f-056566d9b804 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:06:21.887486 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f109c2-aa3f-4a6a-939f-056566d9b804 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:06:21.898543 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fdbb7da-22f0-4590-8299-24dfc78bcbaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:06:21.899645 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fdbb7da-22f0-4590-8299-24dfc78bcbaf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:06:21.899645 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fdbb7da-22f0-4590-8299-24dfc78bcbaf admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:06:21.899934 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fdbb7da-22f0-4590-8299-24dfc78bcbaf admin admin] PATH_INFO: `/v3/users/fb314c7c741d46b4a698a9856ebd9e7b` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:06:21.901248 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fdbb7da-22f0-4590-8299-24dfc78bcbaf admin admin] RBAC: Authorizing `identity:delete_user(user_id=fb314c7c741d46b4a698a9856ebd9e7b)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:06:21.903222 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fdbb7da-22f0-4590-8299-24dfc78bcbaf admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:06:21.930396 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-63f109c2-aa3f-4a6a-939f-056566d9b804 None None] MFA Rules not processed for user `df77ae1fc34247bf975b1de8b3dd2e1e`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 11:06:21.931875 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-63f109c2-aa3f-4a6a-939f-056566d9b804 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:06:21.943832 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 11:06:21.962502 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-9fdbb7da-22f0-4590-8299-24dfc78bcbaf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fb314c7c741d46b4a698a9856ebd9e7b'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:06:21.970702 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-9fdbb7da-22f0-4590-8299-24dfc78bcbaf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fb314c7c741d46b4a698a9856ebd9e7b'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:06:21.971522 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-9fdbb7da-22f0-4590-8299-24dfc78bcbaf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fb314c7c741d46b4a698a9856ebd9e7b'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:06:21.972913 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 11:06:22.000044 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-9fdbb7da-22f0-4590-8299-24dfc78bcbaf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fb314c7c741d46b4a698a9856ebd9e7b'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:06:22.000992 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-9fdbb7da-22f0-4590-8299-24dfc78bcbaf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fb314c7c741d46b4a698a9856ebd9e7b'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:06:22.017049 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1097/2211] 10.222.0.40 () {58 vars in 1224 bytes} [Fri Aug 26 11:06:21 2022] DELETE /identity/v3/users/fb314c7c741d46b4a698a9856ebd9e7b => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 11:06:22.023713 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1115/2212] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 11:06:21 2022] POST /identity/v3/auth/tokens => generated 3349 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 11:06:22.147565 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fdbb7da-22f0-4590-8299-24dfc78bcbaf admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:06:22.151003 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-9fdbb7da-22f0-4590-8299-24dfc78bcbaf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:06:22.246003 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fdbb7da-22f0-4590-8299-24dfc78bcbaf admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:06:22.253170 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c190c3b-9bb3-4c1c-b450-cf1010e527dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:06:22.254027 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c190c3b-9bb3-4c1c-b450-cf1010e527dd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:06:22.254474 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c190c3b-9bb3-4c1c-b450-cf1010e527dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:06:22.255171 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c190c3b-9bb3-4c1c-b450-cf1010e527dd admin admin] PATH_INFO: `/v3/users/81d53661005f47e8a326dd701ca93fb3` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:06:22.256565 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c190c3b-9bb3-4c1c-b450-cf1010e527dd admin admin] RBAC: Authorizing `identity:delete_user(user_id=81d53661005f47e8a326dd701ca93fb3)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:06:22.258781 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c190c3b-9bb3-4c1c-b450-cf1010e527dd admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:06:22.333516 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-3c190c3b-9bb3-4c1c-b450-cf1010e527dd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '81d53661005f47e8a326dd701ca93fb3'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:06:22.349611 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-3c190c3b-9bb3-4c1c-b450-cf1010e527dd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '81d53661005f47e8a326dd701ca93fb3'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:06:22.350722 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-3c190c3b-9bb3-4c1c-b450-cf1010e527dd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '81d53661005f47e8a326dd701ca93fb3'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:06:22.363311 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f109c2-aa3f-4a6a-939f-056566d9b804 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:06:22.366689 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-63f109c2-aa3f-4a6a-939f-056566d9b804 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:06:22.375626 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-3c190c3b-9bb3-4c1c-b450-cf1010e527dd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '81d53661005f47e8a326dd701ca93fb3'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:06:22.376403 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-3c190c3b-9bb3-4c1c-b450-cf1010e527dd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '81d53661005f47e8a326dd701ca93fb3'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:06:22.385804 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f109c2-aa3f-4a6a-939f-056566d9b804 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:06:22.405096 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1098/2213] 10.222.0.40 () {58 vars in 1224 bytes} [Fri Aug 26 11:06:22 2022] DELETE /identity/v3/users/81d53661005f47e8a326dd701ca93fb3 => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 11:06:22.406518 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71e3164-fcb3-4f7f-8134-05066dcb3670 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:06:22.406971 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be81e8f3-c151-47e5-a31d-e19e23c4d181 None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:06:22.407260 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be81e8f3-c151-47e5-a31d-e19e23c4d181 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:06:22.407495 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71e3164-fcb3-4f7f-8134-05066dcb3670 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:06:22.407891 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71e3164-fcb3-4f7f-8134-05066dcb3670 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:06:22.408101 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be81e8f3-c151-47e5-a31d-e19e23c4d181 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:06:22.408338 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71e3164-fcb3-4f7f-8134-05066dcb3670 admin admin] PATH_INFO: `/v3/projects/eeee6cbac36f46b7b04eb2e64ccf9745` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:06:22.409753 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e71e3164-fcb3-4f7f-8134-05066dcb3670 admin admin] RBAC: Authorizing `identity:delete_project(project_id=eeee6cbac36f46b7b04eb2e64ccf9745)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:06:22.412713 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e71e3164-fcb3-4f7f-8134-05066dcb3670 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:06:22.433259 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-be81e8f3-c151-47e5-a31d-e19e23c4d181 None None] MFA Rules not processed for user `ca155a48e2024c32af3fd2e19e5fac11`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 11:06:22.434708 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-be81e8f3-c151-47e5-a31d-e19e23c4d181 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:06:22.475340 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-e71e3164-fcb3-4f7f-8134-05066dcb3670 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eeee6cbac36f46b7b04eb2e64ccf9745', 'request_id': 'req-e71e3164-fcb3-4f7f-8134-05066dcb3670'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:06:22.482745 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-e71e3164-fcb3-4f7f-8134-05066dcb3670 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eeee6cbac36f46b7b04eb2e64ccf9745', 'request_id': 'req-e71e3164-fcb3-4f7f-8134-05066dcb3670'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:06:22.490662 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 11:06:22.491860 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-e71e3164-fcb3-4f7f-8134-05066dcb3670 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eeee6cbac36f46b7b04eb2e64ccf9745', 'request_id': 'req-e71e3164-fcb3-4f7f-8134-05066dcb3670'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:06:22.510353 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1099/2214] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 11:06:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 11:06:22.520014 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-e71e3164-fcb3-4f7f-8134-05066dcb3670 admin admin] The token cache is being invalidate because project eeee6cbac36f46b7b04eb2e64ccf9745 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81424) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 11:06:22.520314 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be81e8f3-c151-47e5-a31d-e19e23c4d181 None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:06:22.520431 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-e71e3164-fcb3-4f7f-8134-05066dcb3670 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:06:22.521832 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-e71e3164-fcb3-4f7f-8134-05066dcb3670 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:06:22.522803 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-be81e8f3-c151-47e5-a31d-e19e23c4d181 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:06:22.523659 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1116/2215] 10.222.0.40 () {58 vars in 1233 bytes} [Fri Aug 26 11:06:22 2022] DELETE /identity/v3/projects/eeee6cbac36f46b7b04eb2e64ccf9745 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 26 11:06:22.530834 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd47d4b1-9ae9-4d21-890e-cc2097d6cb5a None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:06:22.531120 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd47d4b1-9ae9-4d21-890e-cc2097d6cb5a None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:06:22.531395 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd47d4b1-9ae9-4d21-890e-cc2097d6cb5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:06:22.558520 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-bd47d4b1-9ae9-4d21-890e-cc2097d6cb5a None None] MFA Rules not processed for user `ca155a48e2024c32af3fd2e19e5fac11`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 11:06:22.559958 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-bd47d4b1-9ae9-4d21-890e-cc2097d6cb5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:06:22.602580 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be81e8f3-c151-47e5-a31d-e19e23c4d181 None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:06:22.636261 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1117/2216] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 11:06:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 11:06:22.637908 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9be4e20-2bfb-4f53-8244-37dd0298fde4 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:06:22.638318 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9be4e20-2bfb-4f53-8244-37dd0298fde4 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:06:22.638706 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9be4e20-2bfb-4f53-8244-37dd0298fde4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:06:22.667053 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-d9be4e20-2bfb-4f53-8244-37dd0298fde4 None None] MFA Rules not processed for user `ca155a48e2024c32af3fd2e19e5fac11`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 11:06:22.669040 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-d9be4e20-2bfb-4f53-8244-37dd0298fde4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:06:22.679781 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1118/2217] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 11:06:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 11:06:22.684452 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9be4e20-2bfb-4f53-8244-37dd0298fde4 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:06:22.687822 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65eb1df8-7991-4a39-819f-c75b65d4e0d4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91da2acd2534423091537e1ba068d2b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e30ccf0c1204727b786876c141dbcd2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:06:22.688480 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65eb1df8-7991-4a39-819f-c75b65d4e0d4 service neutron] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:06:22.688687 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65eb1df8-7991-4a39-819f-c75b65d4e0d4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:06:22.688958 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65eb1df8-7991-4a39-819f-c75b65d4e0d4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:06:22.690756 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-65eb1df8-7991-4a39-819f-c75b65d4e0d4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:06:22.695042 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9be4e20-2bfb-4f53-8244-37dd0298fde4 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:06:22.700438 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e68584-11ea-4fd2-8b71-de22e334320b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:06:22.701157 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e68584-11ea-4fd2-8b71-de22e334320b admin admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:06:22.701437 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e68584-11ea-4fd2-8b71-de22e334320b admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:06:22.702035 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e68584-11ea-4fd2-8b71-de22e334320b admin admin] PATH_INFO: `/v3/domains` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:06:22.702647 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e68584-11ea-4fd2-8b71-de22e334320b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 11:06:22.702942 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e68584-11ea-4fd2-8b71-de22e334320b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:06:22.704567 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e68584-11ea-4fd2-8b71-de22e334320b admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:06:22.705012 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65eb1df8-7991-4a39-819f-c75b65d4e0d4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:06:22.708446 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65eb1df8-7991-4a39-819f-c75b65d4e0d4 service neutron] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:06:22.727614 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1119/2218] 10.222.0.40 () {58 vars in 1153 bytes} [Fri Aug 26 11:06:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 11:06:22.727815 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1100/2219] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 11:06:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 205 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 26 11:06:22.728670 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e68584-11ea-4fd2-8b71-de22e334320b admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:06:22.739565 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e68584-11ea-4fd2-8b71-de22e334320b admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:06:22.745167 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876db489-57fb-4927-8d7e-e9a4ae17c897 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:06:22.745758 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876db489-57fb-4927-8d7e-e9a4ae17c897 admin admin] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:06:22.746067 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876db489-57fb-4927-8d7e-e9a4ae17c897 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:06:22.746340 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876db489-57fb-4927-8d7e-e9a4ae17c897 admin admin] PATH_INFO: `/v3/projects` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:06:22.746956 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-876db489-57fb-4927-8d7e-e9a4ae17c897 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:06:22.748829 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-876db489-57fb-4927-8d7e-e9a4ae17c897 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:06:22.759586 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 11:06:22.787209 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1120/2220] 10.222.0.40 () {60 vars in 1153 bytes} [Fri Aug 26 11:06:22 2022] POST /identity/v3/projects => generated 399 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 11:06:22.791535 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65eb1df8-7991-4a39-819f-c75b65d4e0d4 service neutron] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:06:22.802119 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65eb1df8-7991-4a39-819f-c75b65d4e0d4 service neutron] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:06:22.889478 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d475c065-646b-4f5e-810b-de497af9d143 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:06:22.890827 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d475c065-646b-4f5e-810b-de497af9d143 admin admin] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:06:22.891290 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d475c065-646b-4f5e-810b-de497af9d143 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:06:22.891682 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d475c065-646b-4f5e-810b-de497af9d143 admin admin] PATH_INFO: `/v3/users` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:06:22.892522 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d475c065-646b-4f5e-810b-de497af9d143 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:06:22.894624 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d475c065-646b-4f5e-810b-de497af9d143 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:06:22.921452 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1101/2221] 10.222.0.40 () {60 vars in 1144 bytes} [Fri Aug 26 11:06:22 2022] POST /identity/v3/users => generated 300 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 26 11:06:22.925553 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876db489-57fb-4927-8d7e-e9a4ae17c897 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:06:22.937405 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876db489-57fb-4927-8d7e-e9a4ae17c897 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:06:22.945800 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38fd5965-dbbe-4e77-a782-7c676df2583f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:06:22.946462 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38fd5965-dbbe-4e77-a782-7c676df2583f admin admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:06:22.946739 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38fd5965-dbbe-4e77-a782-7c676df2583f admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:06:22.947011 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38fd5965-dbbe-4e77-a782-7c676df2583f admin admin] PATH_INFO: `/v3/roles` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:06:22.947662 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38fd5965-dbbe-4e77-a782-7c676df2583f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:06:22.949602 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38fd5965-dbbe-4e77-a782-7c676df2583f admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:06:22.962166 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1121/2222] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 26 11:06:22 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 11:06:22.966126 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d475c065-646b-4f5e-810b-de497af9d143 admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:06:22.982810 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d475c065-646b-4f5e-810b-de497af9d143 admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:06:22.988160 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68293e9b-1178-45cb-887c-71232b826361 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:06:22.988753 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68293e9b-1178-45cb-887c-71232b826361 admin admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:06:22.989033 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68293e9b-1178-45cb-887c-71232b826361 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:06:22.989492 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68293e9b-1178-45cb-887c-71232b826361 admin admin] PATH_INFO: `/v3/roles` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:06:22.990890 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68293e9b-1178-45cb-887c-71232b826361 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:06:22.992752 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68293e9b-1178-45cb-887c-71232b826361 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:06:23.003651 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1102/2223] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 26 11:06:22 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 26 11:06:23.008601 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38fd5965-dbbe-4e77-a782-7c676df2583f admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:06:23.020238 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38fd5965-dbbe-4e77-a782-7c676df2583f admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:06:23.026432 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583ea1e1-59e8-4aac-a083-c3320e901368 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:06:23.027537 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583ea1e1-59e8-4aac-a083-c3320e901368 admin admin] REQUEST_METHOD: `PUT` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:06:23.027859 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583ea1e1-59e8-4aac-a083-c3320e901368 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:06:23.028140 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583ea1e1-59e8-4aac-a083-c3320e901368 admin admin] PATH_INFO: `/v3/projects/4442bde0809f42b4aecb6c60cf9cea29/users/241c84a3981f40d5b7b0ac25518724ef/roles/23e174ce743b46eabe618c212691da55` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:06:23.054932 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-583ea1e1-59e8-4aac-a083-c3320e901368 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4442bde0809f42b4aecb6c60cf9cea29, user_id=241c84a3981f40d5b7b0ac25518724ef, role_id=23e174ce743b46eabe618c212691da55)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:06:23.057639 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-583ea1e1-59e8-4aac-a083-c3320e901368 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:06:23.073998 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1122/2224] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 26 11:06:23 2022] PUT /identity/v3/projects/4442bde0809f42b4aecb6c60cf9cea29/users/241c84a3981f40d5b7b0ac25518724ef/roles/23e174ce743b46eabe618c212691da55 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 11:06:23.085406 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3bcc11-51cd-4a77-aaa7-c944f8799a8d None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:06:23.085713 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3bcc11-51cd-4a77-aaa7-c944f8799a8d None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:06:23.086172 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3bcc11-51cd-4a77-aaa7-c944f8799a8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:06:23.118681 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-3e3bcc11-51cd-4a77-aaa7-c944f8799a8d None None] MFA Rules not processed for user `241c84a3981f40d5b7b0ac25518724ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 11:06:23.122445 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-3e3bcc11-51cd-4a77-aaa7-c944f8799a8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:06:23.140965 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 11:06:23.203803 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1103/2225] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 11:06:23 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 11:06:23.216910 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583ea1e1-59e8-4aac-a083-c3320e901368 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:06:23.221024 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-583ea1e1-59e8-4aac-a083-c3320e901368 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:06:23.254710 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3bcc11-51cd-4a77-aaa7-c944f8799a8d None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:06:23.256916 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-3e3bcc11-51cd-4a77-aaa7-c944f8799a8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:06:23.320959 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583ea1e1-59e8-4aac-a083-c3320e901368 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:06:23.325070 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332caba7-d4e6-4c95-8f2b-6c4589a6deb5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c277338a7f814cc6b01c5419775031bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e30ccf0c1204727b786876c141dbcd2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:06:23.325682 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332caba7-d4e6-4c95-8f2b-6c4589a6deb5 service cinder] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:06:23.325963 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332caba7-d4e6-4c95-8f2b-6c4589a6deb5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:06:23.326655 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332caba7-d4e6-4c95-8f2b-6c4589a6deb5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:06:23.345164 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-332caba7-d4e6-4c95-8f2b-6c4589a6deb5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:06:23.348122 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-332caba7-d4e6-4c95-8f2b-6c4589a6deb5 service cinder] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:06:23.356048 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3bcc11-51cd-4a77-aaa7-c944f8799a8d None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:06:23.360209 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1123/2226] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 11:06:23 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 11:06:23.362616 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f6a45f-5754-4d83-a8f5-24961a96cba3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:06:23.363345 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f6a45f-5754-4d83-a8f5-24961a96cba3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:06:23.363718 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f6a45f-5754-4d83-a8f5-24961a96cba3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:06:23.364083 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f6a45f-5754-4d83-a8f5-24961a96cba3 admin admin] PATH_INFO: `/v3/projects/437b6fe232f14d8e99a03d1cae243561` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:06:23.365601 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87f6a45f-5754-4d83-a8f5-24961a96cba3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=437b6fe232f14d8e99a03d1cae243561)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:06:23.368369 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87f6a45f-5754-4d83-a8f5-24961a96cba3 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:06:23.436850 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-87f6a45f-5754-4d83-a8f5-24961a96cba3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '437b6fe232f14d8e99a03d1cae243561', 'request_id': 'req-87f6a45f-5754-4d83-a8f5-24961a96cba3'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:06:23.445848 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-87f6a45f-5754-4d83-a8f5-24961a96cba3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '437b6fe232f14d8e99a03d1cae243561', 'request_id': 'req-87f6a45f-5754-4d83-a8f5-24961a96cba3'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:06:23.456643 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-87f6a45f-5754-4d83-a8f5-24961a96cba3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '437b6fe232f14d8e99a03d1cae243561', 'request_id': 'req-87f6a45f-5754-4d83-a8f5-24961a96cba3'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:06:23.460640 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332caba7-d4e6-4c95-8f2b-6c4589a6deb5 service cinder] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:06:23.463212 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-332caba7-d4e6-4c95-8f2b-6c4589a6deb5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:06:23.478743 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-87f6a45f-5754-4d83-a8f5-24961a96cba3 admin admin] The token cache is being invalidate because project 437b6fe232f14d8e99a03d1cae243561 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81423) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 11:06:23.479135 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-87f6a45f-5754-4d83-a8f5-24961a96cba3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:06:23.479798 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-87f6a45f-5754-4d83-a8f5-24961a96cba3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:06:23.484962 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332caba7-d4e6-4c95-8f2b-6c4589a6deb5 service cinder] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:06:23.486073 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1104/2227] 10.222.0.40 () {58 vars in 1233 bytes} [Fri Aug 26 11:06:23 2022] DELETE /identity/v3/projects/437b6fe232f14d8e99a03d1cae243561 => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 11:06:23.487357 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24945eb1-ea08-4603-86c0-fd314543e248 None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:06:23.487649 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24945eb1-ea08-4603-86c0-fd314543e248 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:06:23.487918 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24945eb1-ea08-4603-86c0-fd314543e248 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:06:23.516167 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-24945eb1-ea08-4603-86c0-fd314543e248 None None] MFA Rules not processed for user `ca155a48e2024c32af3fd2e19e5fac11`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 11:06:23.517489 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-24945eb1-ea08-4603-86c0-fd314543e248 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:06:23.556461 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5effa336-8b2c-4943-97f5-d07506afef25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:06:23.557101 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5effa336-8b2c-4943-97f5-d07506afef25 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:06:23.557378 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5effa336-8b2c-4943-97f5-d07506afef25 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:06:23.557670 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5effa336-8b2c-4943-97f5-d07506afef25 admin admin] PATH_INFO: `/v3/users/1ca9c5f32486425daec1f85a381e95cc` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:06:23.558821 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5effa336-8b2c-4943-97f5-d07506afef25 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1ca9c5f32486425daec1f85a381e95cc)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:06:23.561369 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5effa336-8b2c-4943-97f5-d07506afef25 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:06:23.576145 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 11:06:23.609643 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1105/2228] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 11:06:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 11:06:23.611983 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90793a3-c31f-4eef-b8c1-b5458c347647 None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:06:23.611983 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90793a3-c31f-4eef-b8c1-b5458c347647 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:06:23.611983 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90793a3-c31f-4eef-b8c1-b5458c347647 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:06:23.635567 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-f90793a3-c31f-4eef-b8c1-b5458c347647 None None] MFA Rules not processed for user `ca155a48e2024c32af3fd2e19e5fac11`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 11:06:23.637083 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-f90793a3-c31f-4eef-b8c1-b5458c347647 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:06:23.638217 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-5effa336-8b2c-4943-97f5-d07506afef25 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1ca9c5f32486425daec1f85a381e95cc'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:06:23.639386 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-5effa336-8b2c-4943-97f5-d07506afef25 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1ca9c5f32486425daec1f85a381e95cc'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:06:23.647706 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-5effa336-8b2c-4943-97f5-d07506afef25 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1ca9c5f32486425daec1f85a381e95cc'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:06:23.649276 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-5effa336-8b2c-4943-97f5-d07506afef25 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1ca9c5f32486425daec1f85a381e95cc'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:06:23.652487 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1106/2229] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 11:06:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 11:06:23.653630 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 11:06:23.657185 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90793a3-c31f-4eef-b8c1-b5458c347647 None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:06:23.668394 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90793a3-c31f-4eef-b8c1-b5458c347647 None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:06:23.669752 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-5effa336-8b2c-4943-97f5-d07506afef25 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1ca9c5f32486425daec1f85a381e95cc'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:06:23.675265 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7104ed90-f423-47c9-b9c6-f6174b9351f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:06:23.676070 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7104ed90-f423-47c9-b9c6-f6174b9351f2 admin admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:06:23.676393 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7104ed90-f423-47c9-b9c6-f6174b9351f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:06:23.676732 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7104ed90-f423-47c9-b9c6-f6174b9351f2 admin admin] PATH_INFO: `/v3/domains` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:06:23.678349 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7104ed90-f423-47c9-b9c6-f6174b9351f2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 11:06:23.678791 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7104ed90-f423-47c9-b9c6-f6174b9351f2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:06:23.680517 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7104ed90-f423-47c9-b9c6-f6174b9351f2 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:06:23.688760 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1124/2230] 10.222.0.40 () {58 vars in 1224 bytes} [Fri Aug 26 11:06:23 2022] DELETE /identity/v3/users/1ca9c5f32486425daec1f85a381e95cc => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 26 11:06:23.697938 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1107/2231] 10.222.0.40 () {58 vars in 1153 bytes} [Fri Aug 26 11:06:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 11:06:23.701944 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5effa336-8b2c-4943-97f5-d07506afef25 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:06:23.793769 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5effa336-8b2c-4943-97f5-d07506afef25 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:06:23.805437 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c735da5-828e-4ab2-b139-e8c7e09154f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:06:23.806135 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c735da5-828e-4ab2-b139-e8c7e09154f7 admin admin] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:06:23.806413 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c735da5-828e-4ab2-b139-e8c7e09154f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:06:23.807044 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c735da5-828e-4ab2-b139-e8c7e09154f7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:06:23.807948 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c735da5-828e-4ab2-b139-e8c7e09154f7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:06:23.812115 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c735da5-828e-4ab2-b139-e8c7e09154f7 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:06:23.884782 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1125/2232] 10.222.0.40 () {60 vars in 1153 bytes} [Fri Aug 26 11:06:23 2022] POST /identity/v3/projects => generated 385 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 11:06:23.887118 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7104ed90-f423-47c9-b9c6-f6174b9351f2 admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:06:23.900617 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7104ed90-f423-47c9-b9c6-f6174b9351f2 admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:06:23.920521 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c735da5-828e-4ab2-b139-e8c7e09154f7 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:06:23.922323 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-2c735da5-828e-4ab2-b139-e8c7e09154f7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:06:23.974498 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54fd6ea7-8bdb-41b6-b717-eda9787c3bd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:06:23.975104 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fd6ea7-8bdb-41b6-b717-eda9787c3bd6 admin admin] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:06:23.975383 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fd6ea7-8bdb-41b6-b717-eda9787c3bd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:06:23.975651 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fd6ea7-8bdb-41b6-b717-eda9787c3bd6 admin admin] PATH_INFO: `/v3/users` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:06:23.976267 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54fd6ea7-8bdb-41b6-b717-eda9787c3bd6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:06:23.978214 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54fd6ea7-8bdb-41b6-b717-eda9787c3bd6 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:06:24.004722 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1108/2233] 10.222.0.40 () {60 vars in 1144 bytes} [Fri Aug 26 11:06:23 2022] POST /identity/v3/users => generated 293 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 11:06:24.005026 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c735da5-828e-4ab2-b139-e8c7e09154f7 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:06:24.006550 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54fd6ea7-8bdb-41b6-b717-eda9787c3bd6 admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:06:24.008854 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db660fef-e607-4ae1-815a-19ed6cd7ab35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:06:24.009793 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db660fef-e607-4ae1-815a-19ed6cd7ab35 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:06:24.010320 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db660fef-e607-4ae1-815a-19ed6cd7ab35 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:06:24.010824 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db660fef-e607-4ae1-815a-19ed6cd7ab35 admin admin] PATH_INFO: `/v3/projects/99e7045fd2644e19b829b93e494b3a51` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:06:24.012949 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db660fef-e607-4ae1-815a-19ed6cd7ab35 admin admin] RBAC: Authorizing `identity:delete_project(project_id=99e7045fd2644e19b829b93e494b3a51)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:06:24.017227 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db660fef-e607-4ae1-815a-19ed6cd7ab35 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:06:24.019105 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54fd6ea7-8bdb-41b6-b717-eda9787c3bd6 admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:06:24.025224 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a492f98-f979-4eea-929a-b7e46b890e76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:06:24.025909 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a492f98-f979-4eea-929a-b7e46b890e76 admin admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:06:24.026227 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a492f98-f979-4eea-929a-b7e46b890e76 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:06:24.026500 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a492f98-f979-4eea-929a-b7e46b890e76 admin admin] PATH_INFO: `/v3/roles` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:06:24.027266 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a492f98-f979-4eea-929a-b7e46b890e76 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:06:24.028768 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a492f98-f979-4eea-929a-b7e46b890e76 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:06:24.043756 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1109/2234] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 26 11:06:24 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 11:06:24.045114 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a492f98-f979-4eea-929a-b7e46b890e76 admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:06:24.056734 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a492f98-f979-4eea-929a-b7e46b890e76 admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:06:24.077504 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-db660fef-e607-4ae1-815a-19ed6cd7ab35 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '99e7045fd2644e19b829b93e494b3a51', 'request_id': 'req-db660fef-e607-4ae1-815a-19ed6cd7ab35'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:06:24.088719 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-db660fef-e607-4ae1-815a-19ed6cd7ab35 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '99e7045fd2644e19b829b93e494b3a51', 'request_id': 'req-db660fef-e607-4ae1-815a-19ed6cd7ab35'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:06:24.094879 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-db660fef-e607-4ae1-815a-19ed6cd7ab35 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '99e7045fd2644e19b829b93e494b3a51', 'request_id': 'req-db660fef-e607-4ae1-815a-19ed6cd7ab35'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:06:24.096744 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=89,keystone:DELETE=9,keystone:INSERT=2 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 11:06:24.116665 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-db660fef-e607-4ae1-815a-19ed6cd7ab35 admin admin] The token cache is being invalidate because project 99e7045fd2644e19b829b93e494b3a51 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81424) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 11:06:24.116665 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-db660fef-e607-4ae1-815a-19ed6cd7ab35 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:06:24.117746 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-db660fef-e607-4ae1-815a-19ed6cd7ab35 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:06:24.119699 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1126/2235] 10.222.0.40 () {58 vars in 1233 bytes} [Fri Aug 26 11:06:23 2022] DELETE /identity/v3/projects/99e7045fd2644e19b829b93e494b3a51 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 26 11:06:24.123900 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db660fef-e607-4ae1-815a-19ed6cd7ab35 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:06:24.125550 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-db660fef-e607-4ae1-815a-19ed6cd7ab35 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:06:24.131066 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401fa91e-f1b6-456f-b81c-1b81d9b76f62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:06:24.131811 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401fa91e-f1b6-456f-b81c-1b81d9b76f62 admin admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:06:24.132155 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401fa91e-f1b6-456f-b81c-1b81d9b76f62 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:06:24.132458 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401fa91e-f1b6-456f-b81c-1b81d9b76f62 admin admin] PATH_INFO: `/v3/roles` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:06:24.134026 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-401fa91e-f1b6-456f-b81c-1b81d9b76f62 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:06:24.134773 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-401fa91e-f1b6-456f-b81c-1b81d9b76f62 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:06:24.145096 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db660fef-e607-4ae1-815a-19ed6cd7ab35 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:06:24.147488 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1110/2236] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 26 11:06:24 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 26 11:06:24.151297 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401fa91e-f1b6-456f-b81c-1b81d9b76f62 admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:06:24.153238 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-401fa91e-f1b6-456f-b81c-1b81d9b76f62 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:06:24.154197 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d974822-6acc-41e5-99e7-0c29dd201d2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:06:24.155060 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d974822-6acc-41e5-99e7-0c29dd201d2b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:06:24.155373 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d974822-6acc-41e5-99e7-0c29dd201d2b admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:06:24.155820 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d974822-6acc-41e5-99e7-0c29dd201d2b admin admin] PATH_INFO: `/v3/users/df77ae1fc34247bf975b1de8b3dd2e1e` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:06:24.157055 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d974822-6acc-41e5-99e7-0c29dd201d2b admin admin] RBAC: Authorizing `identity:delete_user(user_id=df77ae1fc34247bf975b1de8b3dd2e1e)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:06:24.159215 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d974822-6acc-41e5-99e7-0c29dd201d2b admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:06:24.167838 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401fa91e-f1b6-456f-b81c-1b81d9b76f62 admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:06:24.173129 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed3a239-93a6-4758-aacf-d57984aab735 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:06:24.173904 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed3a239-93a6-4758-aacf-d57984aab735 admin admin] REQUEST_METHOD: `PUT` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:06:24.174291 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed3a239-93a6-4758-aacf-d57984aab735 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:06:24.174636 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed3a239-93a6-4758-aacf-d57984aab735 admin admin] PATH_INFO: `/v3/projects/9483fc8d1764471b99c49e4ee46c0b12/users/be8fb67568674d0b91424e2a16407539/roles/23e174ce743b46eabe618c212691da55` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:06:24.193455 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed3a239-93a6-4758-aacf-d57984aab735 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9483fc8d1764471b99c49e4ee46c0b12, user_id=be8fb67568674d0b91424e2a16407539, role_id=23e174ce743b46eabe618c212691da55)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:06:24.196386 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed3a239-93a6-4758-aacf-d57984aab735 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:06:24.212241 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1111/2237] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 26 11:06:24 2022] PUT /identity/v3/projects/9483fc8d1764471b99c49e4ee46c0b12/users/be8fb67568674d0b91424e2a16407539/roles/23e174ce743b46eabe618c212691da55 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 26 11:06:24.217302 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a347aac8-456a-428b-ae3a-0eb4daf3def7 None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:06:24.217586 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a347aac8-456a-428b-ae3a-0eb4daf3def7 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:06:24.217863 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a347aac8-456a-428b-ae3a-0eb4daf3def7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:06:24.221002 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-7d974822-6acc-41e5-99e7-0c29dd201d2b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'df77ae1fc34247bf975b1de8b3dd2e1e'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:06:24.221688 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-7d974822-6acc-41e5-99e7-0c29dd201d2b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'df77ae1fc34247bf975b1de8b3dd2e1e'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:06:24.230139 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-7d974822-6acc-41e5-99e7-0c29dd201d2b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'df77ae1fc34247bf975b1de8b3dd2e1e'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:06:24.230916 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-7d974822-6acc-41e5-99e7-0c29dd201d2b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'df77ae1fc34247bf975b1de8b3dd2e1e'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:06:24.240632 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-a347aac8-456a-428b-ae3a-0eb4daf3def7 None None] MFA Rules not processed for user `ca155a48e2024c32af3fd2e19e5fac11`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 11:06:24.242074 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-a347aac8-456a-428b-ae3a-0eb4daf3def7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:06:24.249066 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-7d974822-6acc-41e5-99e7-0c29dd201d2b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'df77ae1fc34247bf975b1de8b3dd2e1e'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:06:24.261699 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 11:06:24.263436 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1127/2238] 10.222.0.40 () {58 vars in 1224 bytes} [Fri Aug 26 11:06:24 2022] DELETE /identity/v3/users/df77ae1fc34247bf975b1de8b3dd2e1e => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 26 11:06:24.314751 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1112/2239] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 11:06:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 11:06:24.328988 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d974822-6acc-41e5-99e7-0c29dd201d2b admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:06:24.333009 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-7d974822-6acc-41e5-99e7-0c29dd201d2b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:06:24.400806 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d974822-6acc-41e5-99e7-0c29dd201d2b admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:06:24.405006 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ef8c32-47b4-42d6-bc77-57e7cacee685 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91da2acd2534423091537e1ba068d2b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e30ccf0c1204727b786876c141dbcd2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:06:24.405616 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ef8c32-47b4-42d6-bc77-57e7cacee685 service neutron] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:06:24.405893 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ef8c32-47b4-42d6-bc77-57e7cacee685 service neutron] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:06:24.406204 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ef8c32-47b4-42d6-bc77-57e7cacee685 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:06:24.458838 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a347aac8-456a-428b-ae3a-0eb4daf3def7 None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:06:24.460271 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-a347aac8-456a-428b-ae3a-0eb4daf3def7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:06:24.473083 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a347aac8-456a-428b-ae3a-0eb4daf3def7 None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:06:24.489205 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ef8c32-47b4-42d6-bc77-57e7cacee685 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:06:24.491698 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ef8c32-47b4-42d6-bc77-57e7cacee685 service neutron] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:06:24.502213 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1128/2240] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 11:06:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 26 11:06:24.545933 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b3dd6b-3321-4403-ab70-ef85620f40af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:06:24.546584 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b3dd6b-3321-4403-ab70-ef85620f40af admin admin] REQUEST_METHOD: `DELETE` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:06:24.546855 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b3dd6b-3321-4403-ab70-ef85620f40af admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:06:24.547118 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b3dd6b-3321-4403-ab70-ef85620f40af admin admin] PATH_INFO: `/v3/projects/80c6f1ee09494dbcae12bc8a557869c6` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:06:24.548261 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b3dd6b-3321-4403-ab70-ef85620f40af admin admin] RBAC: Authorizing `identity:delete_project(project_id=80c6f1ee09494dbcae12bc8a557869c6)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:06:24.550369 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b3dd6b-3321-4403-ab70-ef85620f40af admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:06:24.604872 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-39b3dd6b-3321-4403-ab70-ef85620f40af admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '80c6f1ee09494dbcae12bc8a557869c6', 'request_id': 'req-39b3dd6b-3321-4403-ab70-ef85620f40af'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:06:24.611154 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-39b3dd6b-3321-4403-ab70-ef85620f40af admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '80c6f1ee09494dbcae12bc8a557869c6', 'request_id': 'req-39b3dd6b-3321-4403-ab70-ef85620f40af'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:06:24.617832 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-39b3dd6b-3321-4403-ab70-ef85620f40af admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '80c6f1ee09494dbcae12bc8a557869c6', 'request_id': 'req-39b3dd6b-3321-4403-ab70-ef85620f40af'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:06:24.631714 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-39b3dd6b-3321-4403-ab70-ef85620f40af admin admin] The token cache is being invalidate because project 80c6f1ee09494dbcae12bc8a557869c6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81423) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 11:06:24.632001 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-39b3dd6b-3321-4403-ab70-ef85620f40af admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:06:24.632674 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-39b3dd6b-3321-4403-ab70-ef85620f40af admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:06:24.634227 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1113/2241] 10.222.0.40 () {58 vars in 1233 bytes} [Fri Aug 26 11:06:24 2022] DELETE /identity/v3/projects/80c6f1ee09494dbcae12bc8a557869c6 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 11:06:24.640210 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dabec73-37e9-4c73-83ae-fcb1132c4299 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:06:24.640676 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dabec73-37e9-4c73-83ae-fcb1132c4299 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:06:24.641053 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dabec73-37e9-4c73-83ae-fcb1132c4299 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:06:24.662863 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-2dabec73-37e9-4c73-83ae-fcb1132c4299 None None] MFA Rules not processed for user `ca155a48e2024c32af3fd2e19e5fac11`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 11:06:24.664201 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-2dabec73-37e9-4c73-83ae-fcb1132c4299 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:06:24.676023 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:INSERT=2,keystone:DELETE=11,keystone:SELECT=87 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 11:06:24.735113 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1129/2242] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 11:06:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 11:06:24.743703 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb5a84a-79c8-4211-8f7e-920017169f1e None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:06:24.744011 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb5a84a-79c8-4211-8f7e-920017169f1e None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:06:24.744308 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb5a84a-79c8-4211-8f7e-920017169f1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:06:24.765413 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-4eb5a84a-79c8-4211-8f7e-920017169f1e None None] MFA Rules not processed for user `ca155a48e2024c32af3fd2e19e5fac11`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 11:06:24.766675 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-4eb5a84a-79c8-4211-8f7e-920017169f1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:06:24.777847 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1114/2243] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 11:06:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 11:06:24.782711 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dabec73-37e9-4c73-83ae-fcb1132c4299 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:06:24.794740 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dabec73-37e9-4c73-83ae-fcb1132c4299 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:06:24.800221 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97443880-1a49-4522-863c-5a425fecbfa2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:06:24.800833 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97443880-1a49-4522-863c-5a425fecbfa2 admin admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:06:24.801246 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97443880-1a49-4522-863c-5a425fecbfa2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:06:24.801525 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97443880-1a49-4522-863c-5a425fecbfa2 admin admin] PATH_INFO: `/v3/domains` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:06:24.802958 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97443880-1a49-4522-863c-5a425fecbfa2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 11:06:24.803342 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97443880-1a49-4522-863c-5a425fecbfa2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:06:24.805098 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97443880-1a49-4522-863c-5a425fecbfa2 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:06:24.821947 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1130/2244] 10.222.0.40 () {58 vars in 1153 bytes} [Fri Aug 26 11:06:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 11:06:24.823280 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb5a84a-79c8-4211-8f7e-920017169f1e None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:06:24.834981 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb5a84a-79c8-4211-8f7e-920017169f1e None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:06:24.843565 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7d79ac-3df3-4394-b06d-7a10bbfae654 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:06:24.844254 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7d79ac-3df3-4394-b06d-7a10bbfae654 admin admin] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:06:24.844600 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7d79ac-3df3-4394-b06d-7a10bbfae654 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:06:24.844901 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7d79ac-3df3-4394-b06d-7a10bbfae654 admin admin] PATH_INFO: `/v3/projects` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:06:24.845732 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b7d79ac-3df3-4394-b06d-7a10bbfae654 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:06:24.847769 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b7d79ac-3df3-4394-b06d-7a10bbfae654 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:06:24.889929 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1115/2245] 10.222.0.40 () {60 vars in 1153 bytes} [Fri Aug 26 11:06:24 2022] POST /identity/v3/projects => generated 355 bytes in 64 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 26 11:06:24.891221 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97443880-1a49-4522-863c-5a425fecbfa2 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:06:24.906960 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97443880-1a49-4522-863c-5a425fecbfa2 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:06:24.974984 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4b3204-8da1-41f8-a84b-d6a9af239a0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:06:24.975643 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4b3204-8da1-41f8-a84b-d6a9af239a0b admin admin] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:06:24.975920 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4b3204-8da1-41f8-a84b-d6a9af239a0b admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:06:24.976188 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4b3204-8da1-41f8-a84b-d6a9af239a0b admin admin] PATH_INFO: `/v3/users` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:06:24.976808 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae4b3204-8da1-41f8-a84b-d6a9af239a0b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:06:24.978739 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae4b3204-8da1-41f8-a84b-d6a9af239a0b admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:06:25.002571 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1131/2246] 10.222.0.40 () {60 vars in 1144 bytes} [Fri Aug 26 11:06:24 2022] POST /identity/v3/users => generated 278 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 11:06:25.009141 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7d79ac-3df3-4394-b06d-7a10bbfae654 admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:06:25.020755 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7d79ac-3df3-4394-b06d-7a10bbfae654 admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:06:25.027582 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e3fd73-8fbb-4d35-a957-3a5b62035082 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:06:25.028302 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e3fd73-8fbb-4d35-a957-3a5b62035082 admin admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:06:25.028669 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e3fd73-8fbb-4d35-a957-3a5b62035082 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:06:25.029034 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e3fd73-8fbb-4d35-a957-3a5b62035082 admin admin] PATH_INFO: `/v3/roles` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:06:25.029766 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e3fd73-8fbb-4d35-a957-3a5b62035082 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:06:25.031654 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e3fd73-8fbb-4d35-a957-3a5b62035082 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:06:25.044203 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1116/2247] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 26 11:06:25 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 11:06:25.047934 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4b3204-8da1-41f8-a84b-d6a9af239a0b admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:06:25.060208 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4b3204-8da1-41f8-a84b-d6a9af239a0b admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:06:25.065908 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa077d6-c4cf-43d5-91f7-863460aa2947 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:06:25.066505 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa077d6-c4cf-43d5-91f7-863460aa2947 admin admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:06:25.066743 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa077d6-c4cf-43d5-91f7-863460aa2947 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:06:25.067007 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa077d6-c4cf-43d5-91f7-863460aa2947 admin admin] PATH_INFO: `/v3/roles` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:06:25.067837 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa077d6-c4cf-43d5-91f7-863460aa2947 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:06:25.069602 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa077d6-c4cf-43d5-91f7-863460aa2947 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:06:25.082307 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1132/2248] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 26 11:06:25 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 26 11:06:25.085692 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e3fd73-8fbb-4d35-a957-3a5b62035082 admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:06:25.107069 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e3fd73-8fbb-4d35-a957-3a5b62035082 admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:06:25.113706 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f64531-0f39-4862-972a-87a95fedd155 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:06:25.114293 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f64531-0f39-4862-972a-87a95fedd155 admin admin] REQUEST_METHOD: `PUT` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:06:25.114588 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f64531-0f39-4862-972a-87a95fedd155 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:06:25.115521 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f64531-0f39-4862-972a-87a95fedd155 admin admin] PATH_INFO: `/v3/projects/f9f7dd353f6c4b6588df43ff02080e84/users/58c3ed60e4e743cb9cdba362e530fe57/roles/23e174ce743b46eabe618c212691da55` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:06:25.148006 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 11:06:25.157182 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f64531-0f39-4862-972a-87a95fedd155 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f9f7dd353f6c4b6588df43ff02080e84, user_id=58c3ed60e4e743cb9cdba362e530fe57, role_id=23e174ce743b46eabe618c212691da55)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:06:25.167517 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f64531-0f39-4862-972a-87a95fedd155 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:06:25.190569 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1117/2249] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 26 11:06:25 2022] PUT /identity/v3/projects/f9f7dd353f6c4b6588df43ff02080e84/users/58c3ed60e4e743cb9cdba362e530fe57/roles/23e174ce743b46eabe618c212691da55 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 26 11:06:25.202010 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686e486d-c5ee-4d89-8d49-dbeca2a8474c None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:06:25.202284 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686e486d-c5ee-4d89-8d49-dbeca2a8474c None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:06:25.202557 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686e486d-c5ee-4d89-8d49-dbeca2a8474c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:06:25.244283 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-686e486d-c5ee-4d89-8d49-dbeca2a8474c None None] MFA Rules not processed for user `58c3ed60e4e743cb9cdba362e530fe57`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 11:06:25.245815 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-686e486d-c5ee-4d89-8d49-dbeca2a8474c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:06:25.360829 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1133/2250] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 11:06:25 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 11:06:25.366923 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f64531-0f39-4862-972a-87a95fedd155 admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:06:25.493615 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f64531-0f39-4862-972a-87a95fedd155 admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:06:25.499091 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d2f6df-5aab-4f94-910b-b5b55a5a35f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:06:25.499992 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d2f6df-5aab-4f94-910b-b5b55a5a35f8 admin admin] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:06:25.499992 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d2f6df-5aab-4f94-910b-b5b55a5a35f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:06:25.500243 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d2f6df-5aab-4f94-910b-b5b55a5a35f8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:06:25.500889 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d2f6df-5aab-4f94-910b-b5b55a5a35f8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:06:25.502846 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d2f6df-5aab-4f94-910b-b5b55a5a35f8 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:06:25.547235 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1118/2251] 10.222.0.40 () {60 vars in 1153 bytes} [Fri Aug 26 11:06:25 2022] POST /identity/v3/projects => generated 355 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 11:06:25.548662 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686e486d-c5ee-4d89-8d49-dbeca2a8474c None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:06:25.560913 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686e486d-c5ee-4d89-8d49-dbeca2a8474c None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:06:25.569326 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 11:06:25.632536 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89795995-ac30-4e15-b7fa-baee3ceea84e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:06:25.633569 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89795995-ac30-4e15-b7fa-baee3ceea84e admin admin] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:06:25.633957 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89795995-ac30-4e15-b7fa-baee3ceea84e admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:06:25.634372 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89795995-ac30-4e15-b7fa-baee3ceea84e admin admin] PATH_INFO: `/v3/users` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:06:25.635213 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89795995-ac30-4e15-b7fa-baee3ceea84e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:06:25.637390 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89795995-ac30-4e15-b7fa-baee3ceea84e admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:06:25.667593 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1134/2252] 10.222.0.40 () {60 vars in 1144 bytes} [Fri Aug 26 11:06:25 2022] POST /identity/v3/users => generated 284 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 11:06:25.671554 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d2f6df-5aab-4f94-910b-b5b55a5a35f8 admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:06:25.686044 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d2f6df-5aab-4f94-910b-b5b55a5a35f8 admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:06:25.690987 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e480d35e-7c37-491e-a14d-7968ba1acf4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:06:25.691594 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e480d35e-7c37-491e-a14d-7968ba1acf4f admin admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:06:25.691876 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e480d35e-7c37-491e-a14d-7968ba1acf4f admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:06:25.692276 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e480d35e-7c37-491e-a14d-7968ba1acf4f admin admin] PATH_INFO: `/v3/roles` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:06:25.693544 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e480d35e-7c37-491e-a14d-7968ba1acf4f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:06:25.695169 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e480d35e-7c37-491e-a14d-7968ba1acf4f admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:06:25.713888 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1119/2253] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 26 11:06:25 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 11:06:25.715097 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89795995-ac30-4e15-b7fa-baee3ceea84e admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:06:25.726040 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89795995-ac30-4e15-b7fa-baee3ceea84e admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:06:25.731978 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14e8fbc-e5b8-4f49-aff5-9cbe80ce623e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:06:25.733035 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14e8fbc-e5b8-4f49-aff5-9cbe80ce623e admin admin] REQUEST_METHOD: `PUT` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:06:25.733375 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14e8fbc-e5b8-4f49-aff5-9cbe80ce623e admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:06:25.733818 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14e8fbc-e5b8-4f49-aff5-9cbe80ce623e admin admin] PATH_INFO: `/v3/projects/6289c42e63794a558c0c3077f8fe784f/users/e12609523a8240bb904f4f463ea63215/roles/1f97e4602e2e4d6498913a81f9ae8d6a` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:06:25.754850 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e14e8fbc-e5b8-4f49-aff5-9cbe80ce623e admin admin] RBAC: Authorizing `identity:create_grant(project_id=6289c42e63794a558c0c3077f8fe784f, user_id=e12609523a8240bb904f4f463ea63215, role_id=1f97e4602e2e4d6498913a81f9ae8d6a)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:06:25.758238 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e14e8fbc-e5b8-4f49-aff5-9cbe80ce623e admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:06:25.775259 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1135/2254] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 26 11:06:25 2022] PUT /identity/v3/projects/6289c42e63794a558c0c3077f8fe784f/users/e12609523a8240bb904f4f463ea63215/roles/1f97e4602e2e4d6498913a81f9ae8d6a => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 11:06:25.779165 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e480d35e-7c37-491e-a14d-7968ba1acf4f admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:06:25.791159 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e480d35e-7c37-491e-a14d-7968ba1acf4f admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:06:25.878126 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128d2aec-8620-4116-a603-b7e44a91b84c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:06:25.878768 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128d2aec-8620-4116-a603-b7e44a91b84c admin admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:06:25.879182 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128d2aec-8620-4116-a603-b7e44a91b84c admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:06:25.879576 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128d2aec-8620-4116-a603-b7e44a91b84c admin admin] PATH_INFO: `/v3/roles` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:06:25.880987 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-128d2aec-8620-4116-a603-b7e44a91b84c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:06:25.882362 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-128d2aec-8620-4116-a603-b7e44a91b84c admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:06:25.894386 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1120/2255] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 26 11:06:25 2022] GET /identity/v3/roles => generated 1420 bytes in 116 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 11:06:25.899417 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14e8fbc-e5b8-4f49-aff5-9cbe80ce623e admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:06:25.914093 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14e8fbc-e5b8-4f49-aff5-9cbe80ce623e admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:06:25.927120 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce504d5-c9d5-43db-a0c5-80bf78ed302e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:06:25.927998 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce504d5-c9d5-43db-a0c5-80bf78ed302e admin admin] REQUEST_METHOD: `PUT` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:06:25.928381 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce504d5-c9d5-43db-a0c5-80bf78ed302e admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:06:25.929564 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce504d5-c9d5-43db-a0c5-80bf78ed302e admin admin] PATH_INFO: `/v3/projects/6289c42e63794a558c0c3077f8fe784f/users/e12609523a8240bb904f4f463ea63215/roles/1f97e4602e2e4d6498913a81f9ae8d6a` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:06:25.933265 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce504d5-c9d5-43db-a0c5-80bf78ed302e admin admin] RBAC: Authorizing `identity:create_grant(project_id=6289c42e63794a558c0c3077f8fe784f, user_id=e12609523a8240bb904f4f463ea63215, role_id=1f97e4602e2e4d6498913a81f9ae8d6a)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:06:25.938061 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce504d5-c9d5-43db-a0c5-80bf78ed302e admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:06:25.952850 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1136/2256] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 26 11:06:25 2022] PUT /identity/v3/projects/6289c42e63794a558c0c3077f8fe784f/users/e12609523a8240bb904f4f463ea63215/roles/1f97e4602e2e4d6498913a81f9ae8d6a => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 11:06:25.968651 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475bac27-4983-4595-a9d5-9b0ea9cc0f55 None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:06:25.968945 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475bac27-4983-4595-a9d5-9b0ea9cc0f55 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:06:25.969225 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475bac27-4983-4595-a9d5-9b0ea9cc0f55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:06:26.004335 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-475bac27-4983-4595-a9d5-9b0ea9cc0f55 None None] MFA Rules not processed for user `e12609523a8240bb904f4f463ea63215`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 11:06:26.005730 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-475bac27-4983-4595-a9d5-9b0ea9cc0f55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:06:26.059536 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 11:06:26.107013 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1121/2257] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 11:06:25 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 11:06:26.120630 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce504d5-c9d5-43db-a0c5-80bf78ed302e admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:06:26.123436 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-5ce504d5-c9d5-43db-a0c5-80bf78ed302e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:06:26.205854 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce504d5-c9d5-43db-a0c5-80bf78ed302e admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:06:26.210750 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ec7f38-ea96-4aab-bd2c-65a936de7790 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c277338a7f814cc6b01c5419775031bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e30ccf0c1204727b786876c141dbcd2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:06:26.211518 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ec7f38-ea96-4aab-bd2c-65a936de7790 service cinder] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:06:26.211902 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ec7f38-ea96-4aab-bd2c-65a936de7790 service cinder] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:06:26.212315 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ec7f38-ea96-4aab-bd2c-65a936de7790 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:06:26.225617 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ec7f38-ea96-4aab-bd2c-65a936de7790 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:06:26.227222 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ec7f38-ea96-4aab-bd2c-65a936de7790 service cinder] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:06:26.241040 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1137/2258] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 11:06:26 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 11:06:26.327840 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475bac27-4983-4595-a9d5-9b0ea9cc0f55 None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:06:26.339295 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475bac27-4983-4595-a9d5-9b0ea9cc0f55 None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:06:26.344716 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d57a7d-36dd-4c0a-a5f7-e93a8d4fb176 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c277338a7f814cc6b01c5419775031bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e30ccf0c1204727b786876c141dbcd2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:06:26.345394 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d57a7d-36dd-4c0a-a5f7-e93a8d4fb176 service cinder] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:06:26.345746 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d57a7d-36dd-4c0a-a5f7-e93a8d4fb176 service cinder] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:06:26.346119 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d57a7d-36dd-4c0a-a5f7-e93a8d4fb176 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:06:26.405860 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d57a7d-36dd-4c0a-a5f7-e93a8d4fb176 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:06:26.407419 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d57a7d-36dd-4c0a-a5f7-e93a8d4fb176 service cinder] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:06:26.418176 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1122/2259] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 11:06:26 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 11:06:28.789116 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46531e73-7988-4f99-b23c-7ff1d3332eb8 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:06:28.793443 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46531e73-7988-4f99-b23c-7ff1d3332eb8 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:06:28.793993 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46531e73-7988-4f99-b23c-7ff1d3332eb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:06:28.833953 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-46531e73-7988-4f99-b23c-7ff1d3332eb8 None None] MFA Rules not processed for user `be8fb67568674d0b91424e2a16407539`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 11:06:28.835493 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-46531e73-7988-4f99-b23c-7ff1d3332eb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:06:28.915431 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 11:06:28.929566 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1138/2260] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 11:06:28 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 11:06:28.934297 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d57a7d-36dd-4c0a-a5f7-e93a8d4fb176 service cinder] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:06:28.935756 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-b4d57a7d-36dd-4c0a-a5f7-e93a8d4fb176 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:06:29.051266 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d57a7d-36dd-4c0a-a5f7-e93a8d4fb176 service cinder] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:06:29.056804 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5974b89-666e-44fb-a695-d2bfd6a0525a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:06:29.057506 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5974b89-666e-44fb-a695-d2bfd6a0525a admin admin] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:06:29.057851 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5974b89-666e-44fb-a695-d2bfd6a0525a admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:06:29.058226 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5974b89-666e-44fb-a695-d2bfd6a0525a admin admin] PATH_INFO: `/v3/projects` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:06:29.058965 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5974b89-666e-44fb-a695-d2bfd6a0525a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:06:29.060929 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5974b89-666e-44fb-a695-d2bfd6a0525a admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:06:29.100146 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1123/2261] 10.222.0.40 () {60 vars in 1153 bytes} [Fri Aug 26 11:06:28 2022] POST /identity/v3/projects => generated 385 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 11:06:29.104429 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46531e73-7988-4f99-b23c-7ff1d3332eb8 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:06:29.116253 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46531e73-7988-4f99-b23c-7ff1d3332eb8 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:06:29.177041 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa64436-bd94-4443-b3b8-6f43ccbed7e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:06:29.177662 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa64436-bd94-4443-b3b8-6f43ccbed7e6 admin admin] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:06:29.177945 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa64436-bd94-4443-b3b8-6f43ccbed7e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:06:29.178260 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa64436-bd94-4443-b3b8-6f43ccbed7e6 admin admin] PATH_INFO: `/v3/users` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:06:29.178886 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa64436-bd94-4443-b3b8-6f43ccbed7e6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:06:29.180814 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa64436-bd94-4443-b3b8-6f43ccbed7e6 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:06:29.206232 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1139/2262] 10.222.0.40 () {60 vars in 1144 bytes} [Fri Aug 26 11:06:29 2022] POST /identity/v3/users => generated 299 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 11:06:29.207810 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5974b89-666e-44fb-a695-d2bfd6a0525a admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:06:29.219405 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5974b89-666e-44fb-a695-d2bfd6a0525a admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:06:29.224868 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ba041c-0382-47a7-b2fe-8f96def2ac8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:06:29.225492 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ba041c-0382-47a7-b2fe-8f96def2ac8d admin admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:06:29.225770 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ba041c-0382-47a7-b2fe-8f96def2ac8d admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:06:29.226074 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ba041c-0382-47a7-b2fe-8f96def2ac8d admin admin] PATH_INFO: `/v3/roles` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:06:29.226726 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ba041c-0382-47a7-b2fe-8f96def2ac8d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:06:29.228333 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ba041c-0382-47a7-b2fe-8f96def2ac8d admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:06:29.240549 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1124/2263] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 26 11:06:29 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 11:06:29.245469 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa64436-bd94-4443-b3b8-6f43ccbed7e6 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:06:29.256996 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa64436-bd94-4443-b3b8-6f43ccbed7e6 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:06:29.261830 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e7e7c7-3e73-43cf-8d7e-d4ecb45cac58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:06:29.262538 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e7e7c7-3e73-43cf-8d7e-d4ecb45cac58 admin admin] REQUEST_METHOD: `PUT` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:06:29.262816 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e7e7c7-3e73-43cf-8d7e-d4ecb45cac58 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:06:29.263090 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e7e7c7-3e73-43cf-8d7e-d4ecb45cac58 admin admin] PATH_INFO: `/v3/projects/7c297d01bde64d81a908e764a8c9f03a/users/6fbc95685c2444668c5ef13fe648daed/roles/1f97e4602e2e4d6498913a81f9ae8d6a` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:06:29.280473 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e7e7c7-3e73-43cf-8d7e-d4ecb45cac58 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7c297d01bde64d81a908e764a8c9f03a, user_id=6fbc95685c2444668c5ef13fe648daed, role_id=1f97e4602e2e4d6498913a81f9ae8d6a)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:06:29.283282 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e7e7c7-3e73-43cf-8d7e-d4ecb45cac58 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:06:29.304715 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1140/2264] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 26 11:06:29 2022] PUT /identity/v3/projects/7c297d01bde64d81a908e764a8c9f03a/users/6fbc95685c2444668c5ef13fe648daed/roles/1f97e4602e2e4d6498913a81f9ae8d6a => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 11:06:29.308808 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ba041c-0382-47a7-b2fe-8f96def2ac8d admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:06:29.321240 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ba041c-0382-47a7-b2fe-8f96def2ac8d admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:06:29.387386 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 11:06:29.402682 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d186cde8-6d9c-4501-9f1a-02028964f4d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:06:29.403537 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d186cde8-6d9c-4501-9f1a-02028964f4d4 admin admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:06:29.403928 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d186cde8-6d9c-4501-9f1a-02028964f4d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:06:29.404312 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d186cde8-6d9c-4501-9f1a-02028964f4d4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:06:29.405127 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d186cde8-6d9c-4501-9f1a-02028964f4d4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:06:29.407381 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d186cde8-6d9c-4501-9f1a-02028964f4d4 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:06:29.418997 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1125/2265] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 26 11:06:29 2022] GET /identity/v3/roles => generated 1420 bytes in 111 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 11:06:29.423105 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e7e7c7-3e73-43cf-8d7e-d4ecb45cac58 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:06:29.435027 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e7e7c7-3e73-43cf-8d7e-d4ecb45cac58 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:06:29.441136 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d57848-1d77-41a1-8d80-a18c4b8c20d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:06:29.441996 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d57848-1d77-41a1-8d80-a18c4b8c20d9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:06:29.442377 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d57848-1d77-41a1-8d80-a18c4b8c20d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:06:29.443526 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d57848-1d77-41a1-8d80-a18c4b8c20d9 admin admin] PATH_INFO: `/v3/projects/7c297d01bde64d81a908e764a8c9f03a/users/6fbc95685c2444668c5ef13fe648daed/roles/1f97e4602e2e4d6498913a81f9ae8d6a` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:06:29.446739 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d57848-1d77-41a1-8d80-a18c4b8c20d9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7c297d01bde64d81a908e764a8c9f03a, user_id=6fbc95685c2444668c5ef13fe648daed, role_id=1f97e4602e2e4d6498913a81f9ae8d6a)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:06:29.450084 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d57848-1d77-41a1-8d80-a18c4b8c20d9 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:06:29.469231 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1141/2266] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 26 11:06:29 2022] PUT /identity/v3/projects/7c297d01bde64d81a908e764a8c9f03a/users/6fbc95685c2444668c5ef13fe648daed/roles/1f97e4602e2e4d6498913a81f9ae8d6a => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 11:06:31.694524 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d186cde8-6d9c-4501-9f1a-02028964f4d4 admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:06:31.708259 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-d186cde8-6d9c-4501-9f1a-02028964f4d4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:06:31.723626 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d186cde8-6d9c-4501-9f1a-02028964f4d4 admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:06:31.906397 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e7ffa6-4fd9-436f-8062-342a38e1f4e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:06:31.906974 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e7ffa6-4fd9-436f-8062-342a38e1f4e7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:06:31.907256 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e7ffa6-4fd9-436f-8062-342a38e1f4e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:06:31.907531 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e7ffa6-4fd9-436f-8062-342a38e1f4e7 admin admin] PATH_INFO: `/v3/users/241c84a3981f40d5b7b0ac25518724ef` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:06:31.908734 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e7ffa6-4fd9-436f-8062-342a38e1f4e7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=241c84a3981f40d5b7b0ac25518724ef)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:06:31.910715 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e7ffa6-4fd9-436f-8062-342a38e1f4e7 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:06:32.024663 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-84e7ffa6-4fd9-436f-8062-342a38e1f4e7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '241c84a3981f40d5b7b0ac25518724ef'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:06:32.032532 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-84e7ffa6-4fd9-436f-8062-342a38e1f4e7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '241c84a3981f40d5b7b0ac25518724ef'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:06:32.033959 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-84e7ffa6-4fd9-436f-8062-342a38e1f4e7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '241c84a3981f40d5b7b0ac25518724ef'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:06:32.054743 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-84e7ffa6-4fd9-436f-8062-342a38e1f4e7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '241c84a3981f40d5b7b0ac25518724ef'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:06:32.056632 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-84e7ffa6-4fd9-436f-8062-342a38e1f4e7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '241c84a3981f40d5b7b0ac25518724ef'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:06:32.079570 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1126/2267] 10.222.0.40 () {58 vars in 1224 bytes} [Fri Aug 26 11:06:31 2022] DELETE /identity/v3/users/241c84a3981f40d5b7b0ac25518724ef => generated 0 bytes in 386 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 11:06:32.084620 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3c47ba-8998-4442-807c-e28fb6305bb0 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:06:32.085633 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3c47ba-8998-4442-807c-e28fb6305bb0 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:06:32.086152 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3c47ba-8998-4442-807c-e28fb6305bb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:06:32.112944 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-ae3c47ba-8998-4442-807c-e28fb6305bb0 None None] MFA Rules not processed for user `ca155a48e2024c32af3fd2e19e5fac11`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 11:06:32.116077 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-ae3c47ba-8998-4442-807c-e28fb6305bb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:06:32.243301 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1142/2268] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 11:06:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 11:06:32.249177 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e7ffa6-4fd9-436f-8062-342a38e1f4e7 admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:06:32.250938 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-84e7ffa6-4fd9-436f-8062-342a38e1f4e7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:06:32.353503 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e7ffa6-4fd9-436f-8062-342a38e1f4e7 admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:06:32.365487 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e099b5a-0557-421d-a0fc-e196c0284710 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91da2acd2534423091537e1ba068d2b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e30ccf0c1204727b786876c141dbcd2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:06:32.366571 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e099b5a-0557-421d-a0fc-e196c0284710 service neutron] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:06:32.366856 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e099b5a-0557-421d-a0fc-e196c0284710 service neutron] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:06:32.367161 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e099b5a-0557-421d-a0fc-e196c0284710 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:06:32.390205 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e099b5a-0557-421d-a0fc-e196c0284710 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:06:32.391200 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e099b5a-0557-421d-a0fc-e196c0284710 service neutron] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:06:32.417870 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1127/2269] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 11:06:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 11:06:33.777019 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3c47ba-8998-4442-807c-e28fb6305bb0 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:06:33.779629 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-ae3c47ba-8998-4442-807c-e28fb6305bb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:06:33.818948 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3c47ba-8998-4442-807c-e28fb6305bb0 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:06:33.837398 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5dceb8-70ba-404e-9196-306d36e2fc3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:06:33.839346 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5dceb8-70ba-404e-9196-306d36e2fc3b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:06:33.839959 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5dceb8-70ba-404e-9196-306d36e2fc3b admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:06:33.840726 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5dceb8-70ba-404e-9196-306d36e2fc3b admin admin] PATH_INFO: `/v3/projects/4442bde0809f42b4aecb6c60cf9cea29` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:06:33.842770 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc5dceb8-70ba-404e-9196-306d36e2fc3b admin admin] RBAC: Authorizing `identity:delete_project(project_id=4442bde0809f42b4aecb6c60cf9cea29)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:06:33.845046 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc5dceb8-70ba-404e-9196-306d36e2fc3b admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:06:33.987781 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=92,keystone:INSERT=5,keystone:DELETE=3 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 11:06:34.000314 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-bc5dceb8-70ba-404e-9196-306d36e2fc3b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4442bde0809f42b4aecb6c60cf9cea29', 'request_id': 'req-bc5dceb8-70ba-404e-9196-306d36e2fc3b'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:06:34.021063 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-bc5dceb8-70ba-404e-9196-306d36e2fc3b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4442bde0809f42b4aecb6c60cf9cea29', 'request_id': 'req-bc5dceb8-70ba-404e-9196-306d36e2fc3b'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:06:34.050730 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-bc5dceb8-70ba-404e-9196-306d36e2fc3b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4442bde0809f42b4aecb6c60cf9cea29', 'request_id': 'req-bc5dceb8-70ba-404e-9196-306d36e2fc3b'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:06:34.076340 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-bc5dceb8-70ba-404e-9196-306d36e2fc3b admin admin] The token cache is being invalidate because project 4442bde0809f42b4aecb6c60cf9cea29 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81424) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 11:06:34.077146 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-bc5dceb8-70ba-404e-9196-306d36e2fc3b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:06:34.078092 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-bc5dceb8-70ba-404e-9196-306d36e2fc3b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:06:34.081075 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1143/2270] 10.222.0.40 () {58 vars in 1233 bytes} [Fri Aug 26 11:06:33 2022] DELETE /identity/v3/projects/4442bde0809f42b4aecb6c60cf9cea29 => generated 0 bytes in 304 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 11:06:35.610868 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cdd9325-952e-4c1f-bb56-1a91de6d01a8 None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:06:35.611299 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cdd9325-952e-4c1f-bb56-1a91de6d01a8 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:06:35.611649 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cdd9325-952e-4c1f-bb56-1a91de6d01a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:06:35.645079 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-6cdd9325-952e-4c1f-bb56-1a91de6d01a8 None None] MFA Rules not processed for user `6fbc95685c2444668c5ef13fe648daed`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 11:06:35.646521 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-6cdd9325-952e-4c1f-bb56-1a91de6d01a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:06:35.650822 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 11:06:35.778384 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1128/2271] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 11:06:35 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 11:06:35.787493 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5dceb8-70ba-404e-9196-306d36e2fc3b admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:06:35.789033 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-bc5dceb8-70ba-404e-9196-306d36e2fc3b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:06:35.885278 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5dceb8-70ba-404e-9196-306d36e2fc3b admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:06:35.890182 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381aa1a7-531b-49ea-8580-0e9d49676c0e service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8631d7ecfd2849daa8694aecccd22a05', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e30ccf0c1204727b786876c141dbcd2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:06:35.890812 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381aa1a7-531b-49ea-8580-0e9d49676c0e service glance] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:06:35.891130 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381aa1a7-531b-49ea-8580-0e9d49676c0e service glance] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:06:35.891594 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381aa1a7-531b-49ea-8580-0e9d49676c0e service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:06:35.893684 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-381aa1a7-531b-49ea-8580-0e9d49676c0e service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:06:36.000117 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-381aa1a7-531b-49ea-8580-0e9d49676c0e service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:06:36.002487 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-381aa1a7-531b-49ea-8580-0e9d49676c0e service glance] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:06:36.028260 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1144/2272] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 11:06:35 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 241 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 11:06:36.165350 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd757a73-6761-4424-aa79-e5da08556dc5 None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:06:36.165639 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd757a73-6761-4424-aa79-e5da08556dc5 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:06:36.165916 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd757a73-6761-4424-aa79-e5da08556dc5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:06:36.187566 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-dd757a73-6761-4424-aa79-e5da08556dc5 None None] MFA Rules not processed for user `50be12ccbd274a8792615e9dd10b61ca`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 11:06:36.188779 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-dd757a73-6761-4424-aa79-e5da08556dc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:06:36.278785 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1129/2273] 10.222.0.40 () {58 vars in 1009 bytes} [Fri Aug 26 11:06:36 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 11:06:36.326091 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381aa1a7-531b-49ea-8580-0e9d49676c0e service glance] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:06:36.327931 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-381aa1a7-531b-49ea-8580-0e9d49676c0e service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:06:36.421590 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381aa1a7-531b-49ea-8580-0e9d49676c0e service glance] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:06:36.428190 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea91f991-0f71-4b43-bd5b-6db694a0aa5d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6529bea296c9482a973ced5fe9f2212f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e30ccf0c1204727b786876c141dbcd2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:06:36.428819 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea91f991-0f71-4b43-bd5b-6db694a0aa5d service swift] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:06:36.429102 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea91f991-0f71-4b43-bd5b-6db694a0aa5d service swift] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:06:36.429414 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea91f991-0f71-4b43-bd5b-6db694a0aa5d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:06:36.438717 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea91f991-0f71-4b43-bd5b-6db694a0aa5d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:06:36.440339 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea91f991-0f71-4b43-bd5b-6db694a0aa5d service swift] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:06:36.450327 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1145/2274] 10.222.0.40 () {60 vars in 1384 bytes} [Fri Aug 26 11:06:36 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 26 11:06:43.426224 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02bd35e1-bd0e-4b1b-9be3-19f8adb1c55a None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:06:43.426902 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02bd35e1-bd0e-4b1b-9be3-19f8adb1c55a None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:06:43.426902 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02bd35e1-bd0e-4b1b-9be3-19f8adb1c55a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:06:43.467738 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-02bd35e1-bd0e-4b1b-9be3-19f8adb1c55a None None] MFA Rules not processed for user `d6dbbabdd88c4c088bc91e6c43531318`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 11:06:43.469239 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-02bd35e1-bd0e-4b1b-9be3-19f8adb1c55a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:06:43.567502 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1130/2275] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 11:06:43 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 11:06:46.446355 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=89,keystone:DELETE=4,keystone:INSERT=1 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 11:06:46.537885 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea91f991-0f71-4b43-bd5b-6db694a0aa5d service swift] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:06:46.540024 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-ea91f991-0f71-4b43-bd5b-6db694a0aa5d service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:06:46.680695 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea91f991-0f71-4b43-bd5b-6db694a0aa5d service swift] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:06:46.688151 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f80acba-dc1d-45d1-a757-4844f1575f06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:06:46.688654 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f80acba-dc1d-45d1-a757-4844f1575f06 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:06:46.688938 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f80acba-dc1d-45d1-a757-4844f1575f06 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:06:46.689307 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f80acba-dc1d-45d1-a757-4844f1575f06 admin admin] PATH_INFO: `/v3/users/53665970ee784c6180c704d1a54c4c95` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:06:46.691142 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f80acba-dc1d-45d1-a757-4844f1575f06 admin admin] RBAC: Authorizing `identity:delete_user(user_id=53665970ee784c6180c704d1a54c4c95)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:06:46.694741 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f80acba-dc1d-45d1-a757-4844f1575f06 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:06:46.796696 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-0f80acba-dc1d-45d1-a757-4844f1575f06 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '53665970ee784c6180c704d1a54c4c95'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:06:46.798410 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-0f80acba-dc1d-45d1-a757-4844f1575f06 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '53665970ee784c6180c704d1a54c4c95'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:06:46.809704 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-0f80acba-dc1d-45d1-a757-4844f1575f06 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '53665970ee784c6180c704d1a54c4c95'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:06:46.810697 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-0f80acba-dc1d-45d1-a757-4844f1575f06 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '53665970ee784c6180c704d1a54c4c95'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:06:46.830680 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-0f80acba-dc1d-45d1-a757-4844f1575f06 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '53665970ee784c6180c704d1a54c4c95'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:06:46.832695 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02bd35e1-bd0e-4b1b-9be3-19f8adb1c55a None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:06:46.836267 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-02bd35e1-bd0e-4b1b-9be3-19f8adb1c55a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:06:46.861684 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02bd35e1-bd0e-4b1b-9be3-19f8adb1c55a None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:06:46.878400 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1bac95-e9d7-4c80-892a-f7d6b0a9e0c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:06:46.880942 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1bac95-e9d7-4c80-892a-f7d6b0a9e0c7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:06:46.882212 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1bac95-e9d7-4c80-892a-f7d6b0a9e0c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:06:46.882715 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1bac95-e9d7-4c80-892a-f7d6b0a9e0c7 admin admin] PATH_INFO: `/v3/users/58c3ed60e4e743cb9cdba362e530fe57` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:06:46.886789 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc1bac95-e9d7-4c80-892a-f7d6b0a9e0c7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=58c3ed60e4e743cb9cdba362e530fe57)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:06:46.889409 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc1bac95-e9d7-4c80-892a-f7d6b0a9e0c7 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:06:46.890606 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1146/2276] 10.222.0.40 () {58 vars in 1224 bytes} [Fri Aug 26 11:06:46 2022] DELETE /identity/v3/users/53665970ee784c6180c704d1a54c4c95 => generated 0 bytes in 354 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 11:06:46.897285 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f08fc0d-5e26-4a44-a01e-d12c63a202a4 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:06:46.897690 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f08fc0d-5e26-4a44-a01e-d12c63a202a4 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:06:46.898127 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f08fc0d-5e26-4a44-a01e-d12c63a202a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:06:46.938080 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-8f08fc0d-5e26-4a44-a01e-d12c63a202a4 None None] MFA Rules not processed for user `ca155a48e2024c32af3fd2e19e5fac11`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 11:06:46.940682 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-8f08fc0d-5e26-4a44-a01e-d12c63a202a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:06:46.999893 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 11:06:47.006364 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-dc1bac95-e9d7-4c80-892a-f7d6b0a9e0c7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '58c3ed60e4e743cb9cdba362e530fe57'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:06:47.022261 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-dc1bac95-e9d7-4c80-892a-f7d6b0a9e0c7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '58c3ed60e4e743cb9cdba362e530fe57'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:06:47.023311 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-dc1bac95-e9d7-4c80-892a-f7d6b0a9e0c7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '58c3ed60e4e743cb9cdba362e530fe57'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:06:47.055716 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-dc1bac95-e9d7-4c80-892a-f7d6b0a9e0c7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '58c3ed60e4e743cb9cdba362e530fe57'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:06:47.056476 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-dc1bac95-e9d7-4c80-892a-f7d6b0a9e0c7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '58c3ed60e4e743cb9cdba362e530fe57'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:06:47.080448 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1131/2277] 10.222.0.40 () {58 vars in 1224 bytes} [Fri Aug 26 11:06:46 2022] DELETE /identity/v3/users/58c3ed60e4e743cb9cdba362e530fe57 => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 26 11:06:47.085703 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15407b04-fba8-480e-b0c5-a078cdf3057e None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:06:47.085993 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15407b04-fba8-480e-b0c5-a078cdf3057e None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:06:47.086272 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15407b04-fba8-480e-b0c5-a078cdf3057e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:06:47.089649 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1147/2278] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 11:06:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 11:06:47.096167 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f08fc0d-5e26-4a44-a01e-d12c63a202a4 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:06:47.100225 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-8f08fc0d-5e26-4a44-a01e-d12c63a202a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:06:47.131709 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 11:06:47.134283 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-15407b04-fba8-480e-b0c5-a078cdf3057e None None] MFA Rules not processed for user `ca155a48e2024c32af3fd2e19e5fac11`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 11:06:47.136631 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-15407b04-fba8-480e-b0c5-a078cdf3057e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:06:47.202341 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f08fc0d-5e26-4a44-a01e-d12c63a202a4 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:06:47.256469 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1132/2279] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 11:06:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 11:06:47.267419 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15407b04-fba8-480e-b0c5-a078cdf3057e None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:06:47.267596 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1176d5-6288-4009-b3dc-1418f07db3d8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91da2acd2534423091537e1ba068d2b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e30ccf0c1204727b786876c141dbcd2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:06:47.268140 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1176d5-6288-4009-b3dc-1418f07db3d8 service neutron] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:06:47.268426 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1176d5-6288-4009-b3dc-1418f07db3d8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:06:47.268693 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1176d5-6288-4009-b3dc-1418f07db3d8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:06:47.279749 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b1176d5-6288-4009-b3dc-1418f07db3d8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:06:47.281717 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15407b04-fba8-480e-b0c5-a078cdf3057e None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:06:47.284377 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b1176d5-6288-4009-b3dc-1418f07db3d8 service neutron] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:06:47.288599 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8309c6e-b1e8-4bb0-992a-e34ba86c5ee5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91da2acd2534423091537e1ba068d2b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e30ccf0c1204727b786876c141dbcd2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:06:47.289362 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8309c6e-b1e8-4bb0-992a-e34ba86c5ee5 service neutron] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:06:47.289807 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8309c6e-b1e8-4bb0-992a-e34ba86c5ee5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:06:47.290107 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8309c6e-b1e8-4bb0-992a-e34ba86c5ee5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:06:47.296386 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1148/2280] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 11:06:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 201 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 26 11:06:47.308731 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8309c6e-b1e8-4bb0-992a-e34ba86c5ee5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:06:47.311675 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8309c6e-b1e8-4bb0-992a-e34ba86c5ee5 service neutron] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:06:47.329862 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1133/2281] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 11:06:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 63 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 11:06:47.879097 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1176d5-6288-4009-b3dc-1418f07db3d8 service neutron] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:06:47.882367 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-2b1176d5-6288-4009-b3dc-1418f07db3d8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:06:47.905462 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1176d5-6288-4009-b3dc-1418f07db3d8 service neutron] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:06:47.918782 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057ad53f-6557-48c4-ac31-e9495d0e1384 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:06:47.918782 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057ad53f-6557-48c4-ac31-e9495d0e1384 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:06:47.918782 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057ad53f-6557-48c4-ac31-e9495d0e1384 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:06:47.918782 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057ad53f-6557-48c4-ac31-e9495d0e1384 admin admin] PATH_INFO: `/v3/projects/f9f7dd353f6c4b6588df43ff02080e84` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:06:47.920803 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-057ad53f-6557-48c4-ac31-e9495d0e1384 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f9f7dd353f6c4b6588df43ff02080e84)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:06:47.922791 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-057ad53f-6557-48c4-ac31-e9495d0e1384 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:06:48.033273 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-057ad53f-6557-48c4-ac31-e9495d0e1384 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f9f7dd353f6c4b6588df43ff02080e84', 'request_id': 'req-057ad53f-6557-48c4-ac31-e9495d0e1384'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:06:48.042824 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-057ad53f-6557-48c4-ac31-e9495d0e1384 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f9f7dd353f6c4b6588df43ff02080e84', 'request_id': 'req-057ad53f-6557-48c4-ac31-e9495d0e1384'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:06:48.049656 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-057ad53f-6557-48c4-ac31-e9495d0e1384 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f9f7dd353f6c4b6588df43ff02080e84', 'request_id': 'req-057ad53f-6557-48c4-ac31-e9495d0e1384'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:06:48.068206 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-057ad53f-6557-48c4-ac31-e9495d0e1384 admin admin] The token cache is being invalidate because project f9f7dd353f6c4b6588df43ff02080e84 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81424) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 11:06:48.068699 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-057ad53f-6557-48c4-ac31-e9495d0e1384 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:06:48.070147 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-057ad53f-6557-48c4-ac31-e9495d0e1384 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:06:48.075493 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1149/2282] 10.222.0.40 () {58 vars in 1233 bytes} [Fri Aug 26 11:06:47 2022] DELETE /identity/v3/projects/f9f7dd353f6c4b6588df43ff02080e84 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 11:06:48.076045 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8309c6e-b1e8-4bb0-992a-e34ba86c5ee5 service neutron] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:06:48.093687 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8309c6e-b1e8-4bb0-992a-e34ba86c5ee5 service neutron] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:06:48.151129 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057ad53f-6557-48c4-ac31-e9495d0e1384 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:06:48.152588 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-057ad53f-6557-48c4-ac31-e9495d0e1384 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:06:48.241785 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa219dd6-2afd-4c98-9745-a5938a1887cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:06:48.242451 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa219dd6-2afd-4c98-9745-a5938a1887cf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:06:48.242730 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa219dd6-2afd-4c98-9745-a5938a1887cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:06:48.248592 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa219dd6-2afd-4c98-9745-a5938a1887cf admin admin] PATH_INFO: `/v3/users/e12609523a8240bb904f4f463ea63215` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:06:48.251442 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa219dd6-2afd-4c98-9745-a5938a1887cf admin admin] RBAC: Authorizing `identity:delete_user(user_id=e12609523a8240bb904f4f463ea63215)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:06:48.257529 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa219dd6-2afd-4c98-9745-a5938a1887cf admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:06:48.314099 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057ad53f-6557-48c4-ac31-e9495d0e1384 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:06:48.323933 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a40d558-0b45-4f57-b6fe-05da9aef2771 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:06:48.324593 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a40d558-0b45-4f57-b6fe-05da9aef2771 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:06:48.325700 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a40d558-0b45-4f57-b6fe-05da9aef2771 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:06:48.326094 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a40d558-0b45-4f57-b6fe-05da9aef2771 admin admin] PATH_INFO: `/v3/projects/9a029ac2e8a446719206c416b51452c0` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:06:48.327648 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a40d558-0b45-4f57-b6fe-05da9aef2771 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9a029ac2e8a446719206c416b51452c0)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:06:48.330657 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a40d558-0b45-4f57-b6fe-05da9aef2771 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:06:48.334896 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 11:06:48.376704 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-aa219dd6-2afd-4c98-9745-a5938a1887cf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e12609523a8240bb904f4f463ea63215'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:06:48.384608 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 11:06:48.392830 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-aa219dd6-2afd-4c98-9745-a5938a1887cf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e12609523a8240bb904f4f463ea63215'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:06:48.393838 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-aa219dd6-2afd-4c98-9745-a5938a1887cf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e12609523a8240bb904f4f463ea63215'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:06:48.421252 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-aa219dd6-2afd-4c98-9745-a5938a1887cf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e12609523a8240bb904f4f463ea63215'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:06:48.424406 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-aa219dd6-2afd-4c98-9745-a5938a1887cf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e12609523a8240bb904f4f463ea63215'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:06:48.429164 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-4a40d558-0b45-4f57-b6fe-05da9aef2771 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9a029ac2e8a446719206c416b51452c0', 'request_id': 'req-4a40d558-0b45-4f57-b6fe-05da9aef2771'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:06:48.439046 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-4a40d558-0b45-4f57-b6fe-05da9aef2771 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9a029ac2e8a446719206c416b51452c0', 'request_id': 'req-4a40d558-0b45-4f57-b6fe-05da9aef2771'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:06:48.452558 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-4a40d558-0b45-4f57-b6fe-05da9aef2771 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9a029ac2e8a446719206c416b51452c0', 'request_id': 'req-4a40d558-0b45-4f57-b6fe-05da9aef2771'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:06:48.457309 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1134/2283] 10.222.0.40 () {58 vars in 1224 bytes} [Fri Aug 26 11:06:48 2022] DELETE /identity/v3/users/e12609523a8240bb904f4f463ea63215 => generated 0 bytes in 382 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 26 11:06:48.483911 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-4a40d558-0b45-4f57-b6fe-05da9aef2771 admin admin] The token cache is being invalidate because project 9a029ac2e8a446719206c416b51452c0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81424) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 11:06:48.484337 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-4a40d558-0b45-4f57-b6fe-05da9aef2771 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:06:48.485089 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-4a40d558-0b45-4f57-b6fe-05da9aef2771 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:06:48.487468 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1150/2284] 10.222.0.40 () {58 vars in 1233 bytes} [Fri Aug 26 11:06:48 2022] DELETE /identity/v3/projects/9a029ac2e8a446719206c416b51452c0 => generated 0 bytes in 337 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 26 11:06:48.494630 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f868e3f8-ec0a-4b1a-bc7e-de9fe5c90be0 None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:06:48.495043 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f868e3f8-ec0a-4b1a-bc7e-de9fe5c90be0 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:06:48.495546 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f868e3f8-ec0a-4b1a-bc7e-de9fe5c90be0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:06:48.525094 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-f868e3f8-ec0a-4b1a-bc7e-de9fe5c90be0 None None] MFA Rules not processed for user `ca155a48e2024c32af3fd2e19e5fac11`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 11:06:48.526638 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-f868e3f8-ec0a-4b1a-bc7e-de9fe5c90be0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:06:48.627571 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1135/2285] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 11:06:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 11:06:48.635300 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbf212c-4c4f-4407-8233-d43ca3f933b6 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:06:48.636099 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbf212c-4c4f-4407-8233-d43ca3f933b6 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:06:48.637003 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbf212c-4c4f-4407-8233-d43ca3f933b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:06:48.671601 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-7fbf212c-4c4f-4407-8233-d43ca3f933b6 None None] MFA Rules not processed for user `ca155a48e2024c32af3fd2e19e5fac11`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 11:06:48.673236 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-7fbf212c-4c4f-4407-8233-d43ca3f933b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:06:48.686351 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1151/2286] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 11:06:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 11:06:48.688436 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f868e3f8-ec0a-4b1a-bc7e-de9fe5c90be0 None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:06:48.702758 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f868e3f8-ec0a-4b1a-bc7e-de9fe5c90be0 None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:06:48.715128 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a7b8c1-da4f-4239-bded-cda7aa6404de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:06:48.716456 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a7b8c1-da4f-4239-bded-cda7aa6404de admin admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:06:48.716949 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a7b8c1-da4f-4239-bded-cda7aa6404de admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:06:48.717292 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a7b8c1-da4f-4239-bded-cda7aa6404de admin admin] PATH_INFO: `/v3/domains` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:06:48.718729 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a7b8c1-da4f-4239-bded-cda7aa6404de admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 11:06:48.719602 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a7b8c1-da4f-4239-bded-cda7aa6404de admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:06:48.721730 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a7b8c1-da4f-4239-bded-cda7aa6404de admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:06:48.744333 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1136/2287] 10.222.0.40 () {58 vars in 1153 bytes} [Fri Aug 26 11:06:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 11:06:48.750964 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fbf212c-4c4f-4407-8233-d43ca3f933b6 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:06:48.753632 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84ba111-6e1c-431d-b275-72bb7a946a7e None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:06:48.754273 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84ba111-6e1c-431d-b275-72bb7a946a7e None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:06:48.754571 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84ba111-6e1c-431d-b275-72bb7a946a7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:06:48.766767 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fbf212c-4c4f-4407-8233-d43ca3f933b6 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:06:48.778126 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2941b8af-d729-469d-a38c-857a873fc223 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:06:48.779163 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2941b8af-d729-469d-a38c-857a873fc223 admin admin] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:06:48.779659 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2941b8af-d729-469d-a38c-857a873fc223 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:06:48.780130 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2941b8af-d729-469d-a38c-857a873fc223 admin admin] PATH_INFO: `/v3/projects` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:06:48.780585 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-a84ba111-6e1c-431d-b275-72bb7a946a7e None None] MFA Rules not processed for user `be8fb67568674d0b91424e2a16407539`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 11:06:48.781224 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2941b8af-d729-469d-a38c-857a873fc223 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:06:48.781841 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-a84ba111-6e1c-431d-b275-72bb7a946a7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:06:48.786367 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2941b8af-d729-469d-a38c-857a873fc223 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:06:48.828922 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1152/2288] 10.222.0.40 () {60 vars in 1153 bytes} [Fri Aug 26 11:06:48 2022] POST /identity/v3/projects => generated 375 bytes in 76 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 11:06:48.831066 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2941b8af-d729-469d-a38c-857a873fc223 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:06:48.833047 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-2941b8af-d729-469d-a38c-857a873fc223 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:06:48.906272 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1137/2289] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 11:06:48 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 11:06:48.910475 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84ba111-6e1c-431d-b275-72bb7a946a7e None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:06:48.927984 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84ba111-6e1c-431d-b275-72bb7a946a7e None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:06:48.933367 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6167a332-3b75-453e-bfd2-915f6bdaf981 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:06:48.934313 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6167a332-3b75-453e-bfd2-915f6bdaf981 admin admin] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:06:48.934949 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6167a332-3b75-453e-bfd2-915f6bdaf981 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:06:48.935434 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6167a332-3b75-453e-bfd2-915f6bdaf981 admin admin] PATH_INFO: `/v3/users` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:06:48.936291 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6167a332-3b75-453e-bfd2-915f6bdaf981 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:06:48.938030 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2941b8af-d729-469d-a38c-857a873fc223 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:06:48.941502 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6167a332-3b75-453e-bfd2-915f6bdaf981 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:06:48.943795 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a30149-d945-47c1-9eb0-e74212151518 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:06:48.944450 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a30149-d945-47c1-9eb0-e74212151518 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:06:48.944871 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a30149-d945-47c1-9eb0-e74212151518 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:06:48.945146 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a30149-d945-47c1-9eb0-e74212151518 admin admin] PATH_INFO: `/v3/projects/6289c42e63794a558c0c3077f8fe784f` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:06:48.946471 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a30149-d945-47c1-9eb0-e74212151518 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6289c42e63794a558c0c3077f8fe784f)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:06:48.948394 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a30149-d945-47c1-9eb0-e74212151518 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:06:48.979174 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1138/2290] 10.222.0.40 () {60 vars in 1144 bytes} [Fri Aug 26 11:06:48 2022] POST /identity/v3/users => generated 288 bytes in 74 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 11:06:48.980268 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6167a332-3b75-453e-bfd2-915f6bdaf981 admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:06:48.996944 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6167a332-3b75-453e-bfd2-915f6bdaf981 admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:06:49.004574 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e9de14-865e-4a07-a91d-d6321f102029 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:06:49.005830 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e9de14-865e-4a07-a91d-d6321f102029 admin admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:06:49.006145 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e9de14-865e-4a07-a91d-d6321f102029 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:06:49.006420 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e9de14-865e-4a07-a91d-d6321f102029 admin admin] PATH_INFO: `/v3/roles` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:06:49.007076 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e9de14-865e-4a07-a91d-d6321f102029 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:06:49.008672 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e9de14-865e-4a07-a91d-d6321f102029 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:06:49.023956 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1139/2291] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 26 11:06:48 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 11:06:49.031410 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-55a30149-d945-47c1-9eb0-e74212151518 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6289c42e63794a558c0c3077f8fe784f', 'request_id': 'req-55a30149-d945-47c1-9eb0-e74212151518'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:06:49.033514 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e9de14-865e-4a07-a91d-d6321f102029 admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:06:49.043015 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-55a30149-d945-47c1-9eb0-e74212151518 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6289c42e63794a558c0c3077f8fe784f', 'request_id': 'req-55a30149-d945-47c1-9eb0-e74212151518'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:06:49.045874 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e9de14-865e-4a07-a91d-d6321f102029 admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:06:49.050260 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-55a30149-d945-47c1-9eb0-e74212151518 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6289c42e63794a558c0c3077f8fe784f', 'request_id': 'req-55a30149-d945-47c1-9eb0-e74212151518'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:06:49.053999 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=84,keystone:DELETE=13,keystone:INSERT=3 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 11:06:49.071618 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-55a30149-d945-47c1-9eb0-e74212151518 admin admin] The token cache is being invalidate because project 6289c42e63794a558c0c3077f8fe784f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81424) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 11:06:49.072595 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-55a30149-d945-47c1-9eb0-e74212151518 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:06:49.073638 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-55a30149-d945-47c1-9eb0-e74212151518 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:06:49.075621 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1153/2292] 10.222.0.40 () {58 vars in 1233 bytes} [Fri Aug 26 11:06:48 2022] DELETE /identity/v3/projects/6289c42e63794a558c0c3077f8fe784f => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 11:06:49.081353 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0024862-31b5-476e-917d-b40532b14c8a None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:06:49.081704 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0024862-31b5-476e-917d-b40532b14c8a None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:06:49.082074 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0024862-31b5-476e-917d-b40532b14c8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:06:49.082571 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=93,keystone:DELETE=3,keystone:INSERT=4 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 11:06:49.105343 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-d0024862-31b5-476e-917d-b40532b14c8a None None] MFA Rules not processed for user `ca155a48e2024c32af3fd2e19e5fac11`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 11:06:49.106699 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-d0024862-31b5-476e-917d-b40532b14c8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:06:49.120734 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7953d208-f481-45c8-a6ac-52c559af60bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:06:49.121502 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7953d208-f481-45c8-a6ac-52c559af60bc admin admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:06:49.121793 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7953d208-f481-45c8-a6ac-52c559af60bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:06:49.122100 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7953d208-f481-45c8-a6ac-52c559af60bc admin admin] PATH_INFO: `/v3/roles` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:06:49.122767 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7953d208-f481-45c8-a6ac-52c559af60bc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:06:49.124381 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7953d208-f481-45c8-a6ac-52c559af60bc admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:06:49.138197 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1140/2293] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 26 11:06:49 2022] GET /identity/v3/roles => generated 1420 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 11:06:49.142345 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7953d208-f481-45c8-a6ac-52c559af60bc admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:06:49.148631 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-7953d208-f481-45c8-a6ac-52c559af60bc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:06:49.164132 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7953d208-f481-45c8-a6ac-52c559af60bc admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:06:49.169603 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821e1006-8686-45ab-8fc9-6104ef0939fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:06:49.170306 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821e1006-8686-45ab-8fc9-6104ef0939fa admin admin] REQUEST_METHOD: `PUT` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:06:49.170594 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821e1006-8686-45ab-8fc9-6104ef0939fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:06:49.170876 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821e1006-8686-45ab-8fc9-6104ef0939fa admin admin] PATH_INFO: `/v3/projects/1b87c51628c342cfa953a7110735d469/users/275df87a6e384102be2194b74b180358/roles/23e174ce743b46eabe618c212691da55` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:06:49.193874 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1154/2294] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 11:06:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 11:06:49.197421 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-821e1006-8686-45ab-8fc9-6104ef0939fa admin admin] RBAC: Authorizing `identity:create_grant(project_id=1b87c51628c342cfa953a7110735d469, user_id=275df87a6e384102be2194b74b180358, role_id=23e174ce743b46eabe618c212691da55)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:06:49.200267 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-821e1006-8686-45ab-8fc9-6104ef0939fa admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:06:49.201671 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13ef073-ffd9-4a98-9074-48b539268bcc None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:06:49.202080 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13ef073-ffd9-4a98-9074-48b539268bcc None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:06:49.202639 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13ef073-ffd9-4a98-9074-48b539268bcc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:06:49.216383 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1141/2295] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 26 11:06:49 2022] PUT /identity/v3/projects/1b87c51628c342cfa953a7110735d469/users/275df87a6e384102be2194b74b180358/roles/23e174ce743b46eabe618c212691da55 => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 26 11:06:49.225170 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-a13ef073-ffd9-4a98-9074-48b539268bcc None None] MFA Rules not processed for user `ca155a48e2024c32af3fd2e19e5fac11`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 11:06:49.227003 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-a13ef073-ffd9-4a98-9074-48b539268bcc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:06:49.227869 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fff5e5-a01a-4b5e-9d5a-ec2e80199a31 None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:06:49.228712 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fff5e5-a01a-4b5e-9d5a-ec2e80199a31 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:06:49.228712 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fff5e5-a01a-4b5e-9d5a-ec2e80199a31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:06:49.275266 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-26fff5e5-a01a-4b5e-9d5a-ec2e80199a31 None None] MFA Rules not processed for user `275df87a6e384102be2194b74b180358`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 11:06:49.277332 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-26fff5e5-a01a-4b5e-9d5a-ec2e80199a31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:06:49.299095 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1155/2296] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 11:06:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 11:06:49.303881 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13ef073-ffd9-4a98-9074-48b539268bcc None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:06:49.313680 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13ef073-ffd9-4a98-9074-48b539268bcc None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:06:49.319449 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f094a00-ff2f-4212-a16e-0dae6d812d44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:06:49.320050 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f094a00-ff2f-4212-a16e-0dae6d812d44 admin admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:06:49.320328 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f094a00-ff2f-4212-a16e-0dae6d812d44 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:06:49.320596 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f094a00-ff2f-4212-a16e-0dae6d812d44 admin admin] PATH_INFO: `/v3/domains` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:06:49.321240 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f094a00-ff2f-4212-a16e-0dae6d812d44 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 11:06:49.321567 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f094a00-ff2f-4212-a16e-0dae6d812d44 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:06:49.323248 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f094a00-ff2f-4212-a16e-0dae6d812d44 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:06:49.339234 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1156/2297] 10.222.0.40 () {58 vars in 1153 bytes} [Fri Aug 26 11:06:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 11:06:49.340772 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f094a00-ff2f-4212-a16e-0dae6d812d44 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:06:49.352704 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f094a00-ff2f-4212-a16e-0dae6d812d44 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:06:49.358197 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b68bf9b-1306-40b2-9654-b9455522c11d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:06:49.358806 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b68bf9b-1306-40b2-9654-b9455522c11d admin admin] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:06:49.359083 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b68bf9b-1306-40b2-9654-b9455522c11d admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:06:49.359355 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b68bf9b-1306-40b2-9654-b9455522c11d admin admin] PATH_INFO: `/v3/projects` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:06:49.360537 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b68bf9b-1306-40b2-9654-b9455522c11d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:06:49.362918 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b68bf9b-1306-40b2-9654-b9455522c11d admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:06:49.367774 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1142/2298] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 11:06:49 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 11:06:49.378754 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26fff5e5-a01a-4b5e-9d5a-ec2e80199a31 None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:06:49.380470 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-26fff5e5-a01a-4b5e-9d5a-ec2e80199a31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:06:49.414086 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1157/2299] 10.222.0.40 () {60 vars in 1153 bytes} [Fri Aug 26 11:06:49 2022] POST /identity/v3/projects => generated 393 bytes in 69 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 26 11:06:49.415067 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b68bf9b-1306-40b2-9654-b9455522c11d admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:06:49.466235 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26fff5e5-a01a-4b5e-9d5a-ec2e80199a31 None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:06:49.513193 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b68bf9b-1306-40b2-9654-b9455522c11d admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:06:49.518595 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3737ed-a1e9-4252-b00f-2eb90c369f86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:06:49.519431 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3737ed-a1e9-4252-b00f-2eb90c369f86 admin admin] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:06:49.519879 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3737ed-a1e9-4252-b00f-2eb90c369f86 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:06:49.520316 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3737ed-a1e9-4252-b00f-2eb90c369f86 admin admin] PATH_INFO: `/v3/users` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:06:49.521887 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd3737ed-a1e9-4252-b00f-2eb90c369f86 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:06:49.524043 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd3737ed-a1e9-4252-b00f-2eb90c369f86 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:06:49.532301 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:DELETE=1,keystone:SELECT=98,keystone:INSERT=1 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 11:06:49.535354 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff111134-64b8-4124-bdff-294c999f0ba3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c277338a7f814cc6b01c5419775031bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e30ccf0c1204727b786876c141dbcd2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:06:49.536710 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff111134-64b8-4124-bdff-294c999f0ba3 service cinder] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:06:49.537447 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff111134-64b8-4124-bdff-294c999f0ba3 service cinder] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:06:49.537798 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff111134-64b8-4124-bdff-294c999f0ba3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:06:49.545147 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 11:06:49.563266 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1158/2300] 10.222.0.40 () {60 vars in 1144 bytes} [Fri Aug 26 11:06:49 2022] POST /identity/v3/users => generated 297 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 11:06:49.571900 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3737ed-a1e9-4252-b00f-2eb90c369f86 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:06:49.587530 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3737ed-a1e9-4252-b00f-2eb90c369f86 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:06:49.604283 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d62bc8c-ef4d-4eee-9c14-101bf7c6df3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:06:49.604905 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d62bc8c-ef4d-4eee-9c14-101bf7c6df3e admin admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:06:49.605181 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d62bc8c-ef4d-4eee-9c14-101bf7c6df3e admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:06:49.605451 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d62bc8c-ef4d-4eee-9c14-101bf7c6df3e admin admin] PATH_INFO: `/v3/roles` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:06:49.606126 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d62bc8c-ef4d-4eee-9c14-101bf7c6df3e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:06:49.607707 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d62bc8c-ef4d-4eee-9c14-101bf7c6df3e admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:06:49.622538 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1159/2301] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 26 11:06:49 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 11:06:49.623765 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff111134-64b8-4124-bdff-294c999f0ba3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:06:49.623977 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d62bc8c-ef4d-4eee-9c14-101bf7c6df3e admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:06:49.625201 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff111134-64b8-4124-bdff-294c999f0ba3 service cinder] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:06:49.637427 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d62bc8c-ef4d-4eee-9c14-101bf7c6df3e admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:06:49.642399 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1143/2302] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 11:06:49 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 264 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 11:06:49.645188 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12a1eb0-4cfd-4b4a-ae58-9fd63ca8b05d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:06:49.645938 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12a1eb0-4cfd-4b4a-ae58-9fd63ca8b05d admin admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:06:49.646269 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12a1eb0-4cfd-4b4a-ae58-9fd63ca8b05d admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:06:49.646538 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12a1eb0-4cfd-4b4a-ae58-9fd63ca8b05d admin admin] PATH_INFO: `/v3/roles` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:06:49.647210 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b12a1eb0-4cfd-4b4a-ae58-9fd63ca8b05d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:06:49.649045 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b12a1eb0-4cfd-4b4a-ae58-9fd63ca8b05d admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:06:49.667305 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1160/2303] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 26 11:06:49 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 26 11:06:49.667717 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff111134-64b8-4124-bdff-294c999f0ba3 service cinder] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:06:49.682103 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff111134-64b8-4124-bdff-294c999f0ba3 service cinder] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:06:49.686789 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51176c6c-68c8-4fd1-91f5-57adfff96d7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:06:49.687611 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51176c6c-68c8-4fd1-91f5-57adfff96d7a admin admin] REQUEST_METHOD: `PUT` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:06:49.687956 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51176c6c-68c8-4fd1-91f5-57adfff96d7a admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:06:49.688294 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51176c6c-68c8-4fd1-91f5-57adfff96d7a admin admin] PATH_INFO: `/v3/projects/0b56b31c31ba4121aa2271ca045eb40c/users/bbcf2dc3345d4412a1a0edf92538b20b/roles/23e174ce743b46eabe618c212691da55` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:06:49.712382 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51176c6c-68c8-4fd1-91f5-57adfff96d7a admin admin] RBAC: Authorizing `identity:create_grant(project_id=0b56b31c31ba4121aa2271ca045eb40c, user_id=bbcf2dc3345d4412a1a0edf92538b20b, role_id=23e174ce743b46eabe618c212691da55)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:06:49.716508 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51176c6c-68c8-4fd1-91f5-57adfff96d7a admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:06:49.735687 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1144/2304] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 26 11:06:49 2022] PUT /identity/v3/projects/0b56b31c31ba4121aa2271ca045eb40c/users/bbcf2dc3345d4412a1a0edf92538b20b/roles/23e174ce743b46eabe618c212691da55 => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 26 11:06:49.751479 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dedfe125-b7d8-4fd7-9ba4-e7f5b083bd0d None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:06:49.752031 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dedfe125-b7d8-4fd7-9ba4-e7f5b083bd0d None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:06:49.752513 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dedfe125-b7d8-4fd7-9ba4-e7f5b083bd0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:06:49.793791 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-dedfe125-b7d8-4fd7-9ba4-e7f5b083bd0d None None] MFA Rules not processed for user `bbcf2dc3345d4412a1a0edf92538b20b`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 11:06:49.795094 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-dedfe125-b7d8-4fd7-9ba4-e7f5b083bd0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:06:49.888763 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1161/2305] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 11:06:49 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 11:06:49.893737 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51176c6c-68c8-4fd1-91f5-57adfff96d7a admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:06:50.012465 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51176c6c-68c8-4fd1-91f5-57adfff96d7a admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:06:50.033641 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c254e558-7e22-454c-9bde-8765a6d33078 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:06:50.034378 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c254e558-7e22-454c-9bde-8765a6d33078 admin admin] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:06:50.034709 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c254e558-7e22-454c-9bde-8765a6d33078 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:06:50.035287 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c254e558-7e22-454c-9bde-8765a6d33078 admin admin] PATH_INFO: `/v3/projects` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:06:50.036016 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c254e558-7e22-454c-9bde-8765a6d33078 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:06:50.042324 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c254e558-7e22-454c-9bde-8765a6d33078 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:06:50.084711 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1145/2306] 10.222.0.40 () {60 vars in 1153 bytes} [Fri Aug 26 11:06:49 2022] POST /identity/v3/projects => generated 391 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 11:06:50.087013 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dedfe125-b7d8-4fd7-9ba4-e7f5b083bd0d None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:06:50.099849 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dedfe125-b7d8-4fd7-9ba4-e7f5b083bd0d None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:06:50.182219 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a3505a-771d-4fe7-bc7b-efc289193d55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:06:50.182219 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a3505a-771d-4fe7-bc7b-efc289193d55 admin admin] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:06:50.182219 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a3505a-771d-4fe7-bc7b-efc289193d55 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:06:50.182219 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a3505a-771d-4fe7-bc7b-efc289193d55 admin admin] PATH_INFO: `/v3/users` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:06:50.183232 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a3505a-771d-4fe7-bc7b-efc289193d55 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:06:50.184695 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a3505a-771d-4fe7-bc7b-efc289193d55 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:06:50.218138 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1162/2307] 10.222.0.40 () {60 vars in 1144 bytes} [Fri Aug 26 11:06:50 2022] POST /identity/v3/users => generated 302 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 11:06:50.222974 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c254e558-7e22-454c-9bde-8765a6d33078 admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:06:50.242481 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c254e558-7e22-454c-9bde-8765a6d33078 admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:06:50.247827 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eaf910f-7d5f-40fb-8db0-b3ea2e20f883 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:06:50.248827 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eaf910f-7d5f-40fb-8db0-b3ea2e20f883 admin admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:06:50.249111 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eaf910f-7d5f-40fb-8db0-b3ea2e20f883 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:06:50.249379 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eaf910f-7d5f-40fb-8db0-b3ea2e20f883 admin admin] PATH_INFO: `/v3/roles` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:06:50.250046 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eaf910f-7d5f-40fb-8db0-b3ea2e20f883 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:06:50.251639 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eaf910f-7d5f-40fb-8db0-b3ea2e20f883 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:06:50.263140 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1146/2308] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 26 11:06:50 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 26 11:06:50.267560 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a3505a-771d-4fe7-bc7b-efc289193d55 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:06:50.281762 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a3505a-771d-4fe7-bc7b-efc289193d55 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:06:50.287519 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd25612-3379-4030-8da0-d6fcb00a0e52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:06:50.288297 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd25612-3379-4030-8da0-d6fcb00a0e52 admin admin] REQUEST_METHOD: `PUT` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:06:50.288715 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd25612-3379-4030-8da0-d6fcb00a0e52 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:06:50.288844 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd25612-3379-4030-8da0-d6fcb00a0e52 admin admin] PATH_INFO: `/v3/projects/6e49f1020e6a4b98b4882b0bae053da4/users/44bb554f1bfa4858a7f93c9a2e12cee3/roles/1f97e4602e2e4d6498913a81f9ae8d6a` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:06:50.308832 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd25612-3379-4030-8da0-d6fcb00a0e52 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6e49f1020e6a4b98b4882b0bae053da4, user_id=44bb554f1bfa4858a7f93c9a2e12cee3, role_id=1f97e4602e2e4d6498913a81f9ae8d6a)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:06:50.311674 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd25612-3379-4030-8da0-d6fcb00a0e52 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:06:50.328291 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1163/2309] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 26 11:06:50 2022] PUT /identity/v3/projects/6e49f1020e6a4b98b4882b0bae053da4/users/44bb554f1bfa4858a7f93c9a2e12cee3/roles/1f97e4602e2e4d6498913a81f9ae8d6a => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 11:06:50.332475 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eaf910f-7d5f-40fb-8db0-b3ea2e20f883 admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:06:50.343743 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eaf910f-7d5f-40fb-8db0-b3ea2e20f883 admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:06:50.404306 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 11:06:50.414774 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a6d2e7-1123-4ef3-a0b6-115943a714e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:06:50.415385 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a6d2e7-1123-4ef3-a0b6-115943a714e8 admin admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:06:50.416043 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a6d2e7-1123-4ef3-a0b6-115943a714e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:06:50.416043 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a6d2e7-1123-4ef3-a0b6-115943a714e8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:06:50.416692 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a6d2e7-1123-4ef3-a0b6-115943a714e8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:06:50.418336 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a6d2e7-1123-4ef3-a0b6-115943a714e8 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:06:50.430429 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1147/2310] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 26 11:06:50 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 26 11:06:50.434445 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd25612-3379-4030-8da0-d6fcb00a0e52 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:06:50.448811 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd25612-3379-4030-8da0-d6fcb00a0e52 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:06:50.461815 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9ecd3d-7c1c-4970-88e0-c3139e942439 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:06:50.463133 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9ecd3d-7c1c-4970-88e0-c3139e942439 admin admin] REQUEST_METHOD: `PUT` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:06:50.463676 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9ecd3d-7c1c-4970-88e0-c3139e942439 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:06:50.464222 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9ecd3d-7c1c-4970-88e0-c3139e942439 admin admin] PATH_INFO: `/v3/projects/6e49f1020e6a4b98b4882b0bae053da4/users/44bb554f1bfa4858a7f93c9a2e12cee3/roles/1f97e4602e2e4d6498913a81f9ae8d6a` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:06:50.467966 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba9ecd3d-7c1c-4970-88e0-c3139e942439 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6e49f1020e6a4b98b4882b0bae053da4, user_id=44bb554f1bfa4858a7f93c9a2e12cee3, role_id=1f97e4602e2e4d6498913a81f9ae8d6a)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:06:50.472686 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba9ecd3d-7c1c-4970-88e0-c3139e942439 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:06:50.493201 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1164/2311] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 26 11:06:50 2022] PUT /identity/v3/projects/6e49f1020e6a4b98b4882b0bae053da4/users/44bb554f1bfa4858a7f93c9a2e12cee3/roles/1f97e4602e2e4d6498913a81f9ae8d6a => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 11:06:50.513425 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f1abe7-4a3e-492e-8a56-30ad29f6fbbc None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:06:50.513830 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f1abe7-4a3e-492e-8a56-30ad29f6fbbc None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:06:50.514856 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f1abe7-4a3e-492e-8a56-30ad29f6fbbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:06:50.577656 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-d7f1abe7-4a3e-492e-8a56-30ad29f6fbbc None None] MFA Rules not processed for user `44bb554f1bfa4858a7f93c9a2e12cee3`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 11:06:50.580876 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-d7f1abe7-4a3e-492e-8a56-30ad29f6fbbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:06:50.715103 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1148/2312] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 11:06:50 2022] POST /identity/v3/auth/tokens => generated 3335 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 11:06:50.733896 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9ecd3d-7c1c-4970-88e0-c3139e942439 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:06:50.752091 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:INSERT=8,keystone:SELECT=92 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 11:06:50.836474 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9ecd3d-7c1c-4970-88e0-c3139e942439 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:06:50.842737 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acfbb824-7fe9-42ad-b0d3-cbb2b6f19fee service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c277338a7f814cc6b01c5419775031bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e30ccf0c1204727b786876c141dbcd2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:06:50.843605 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfbb824-7fe9-42ad-b0d3-cbb2b6f19fee service cinder] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:06:50.844014 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfbb824-7fe9-42ad-b0d3-cbb2b6f19fee service cinder] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:06:50.844413 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfbb824-7fe9-42ad-b0d3-cbb2b6f19fee service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:06:50.865192 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acfbb824-7fe9-42ad-b0d3-cbb2b6f19fee service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:06:50.868060 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acfbb824-7fe9-42ad-b0d3-cbb2b6f19fee service cinder] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:06:50.887324 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1165/2313] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 11:06:50 2022] GET /identity/v3/auth/tokens => generated 3335 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 11:06:50.971393 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f1abe7-4a3e-492e-8a56-30ad29f6fbbc None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:06:50.973602 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-d7f1abe7-4a3e-492e-8a56-30ad29f6fbbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:06:50.998982 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f1abe7-4a3e-492e-8a56-30ad29f6fbbc None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:06:51.006863 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871b5091-14e3-4ca8-bc23-3d38efd63e19 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c277338a7f814cc6b01c5419775031bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e30ccf0c1204727b786876c141dbcd2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:06:51.007988 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871b5091-14e3-4ca8-bc23-3d38efd63e19 service cinder] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:06:51.008952 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871b5091-14e3-4ca8-bc23-3d38efd63e19 service cinder] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:06:51.008952 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871b5091-14e3-4ca8-bc23-3d38efd63e19 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:06:51.174258 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c052e1d6-2658-499b-a837-86f80a9ee394 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:06:51.175024 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c052e1d6-2658-499b-a837-86f80a9ee394 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:06:51.175308 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c052e1d6-2658-499b-a837-86f80a9ee394 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:06:51.202174 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-871b5091-14e3-4ca8-bc23-3d38efd63e19 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:06:51.205392 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-871b5091-14e3-4ca8-bc23-3d38efd63e19 service cinder] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:06:51.208956 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-c052e1d6-2658-499b-a837-86f80a9ee394 None None] MFA Rules not processed for user `275df87a6e384102be2194b74b180358`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 11:06:51.210526 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-c052e1d6-2658-499b-a837-86f80a9ee394 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:06:51.226400 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1149/2314] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 11:06:50 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 256 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 11:06:51.357780 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1166/2315] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 11:06:51 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 11:06:52.551170 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8567e39-6ece-48b9-b8c6-81ea966d4217 None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:06:52.551809 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8567e39-6ece-48b9-b8c6-81ea966d4217 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:06:52.551809 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8567e39-6ece-48b9-b8c6-81ea966d4217 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:06:52.576990 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-a8567e39-6ece-48b9-b8c6-81ea966d4217 None None] MFA Rules not processed for user `50be12ccbd274a8792615e9dd10b61ca`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 11:06:52.578343 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-a8567e39-6ece-48b9-b8c6-81ea966d4217 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:06:52.707661 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1150/2316] 10.222.0.40 () {58 vars in 1009 bytes} [Fri Aug 26 11:06:52 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 11:06:52.722745 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c052e1d6-2658-499b-a837-86f80a9ee394 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:06:52.724452 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-c052e1d6-2658-499b-a837-86f80a9ee394 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:06:52.847504 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c052e1d6-2658-499b-a837-86f80a9ee394 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:06:52.859849 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a23f6d-8e7d-4f3d-ad26-3bbefb78c098 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6529bea296c9482a973ced5fe9f2212f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e30ccf0c1204727b786876c141dbcd2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:06:52.860497 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a23f6d-8e7d-4f3d-ad26-3bbefb78c098 service swift] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:06:52.860827 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a23f6d-8e7d-4f3d-ad26-3bbefb78c098 service swift] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:06:52.861086 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a23f6d-8e7d-4f3d-ad26-3bbefb78c098 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:06:52.882388 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a23f6d-8e7d-4f3d-ad26-3bbefb78c098 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:06:52.884014 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a23f6d-8e7d-4f3d-ad26-3bbefb78c098 service swift] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:06:52.894888 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1167/2317] 10.222.0.40 () {60 vars in 1384 bytes} [Fri Aug 26 11:06:52 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 173 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 26 11:06:56.267820 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8567e39-6ece-48b9-b8c6-81ea966d4217 None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:06:56.293087 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 11:06:56.346677 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8567e39-6ece-48b9-b8c6-81ea966d4217 None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:06:56.351940 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ec8b36-27f5-4336-9c1f-0b9cfa023f64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:06:56.352565 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ec8b36-27f5-4336-9c1f-0b9cfa023f64 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:06:56.352844 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ec8b36-27f5-4336-9c1f-0b9cfa023f64 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:06:56.353110 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ec8b36-27f5-4336-9c1f-0b9cfa023f64 admin admin] PATH_INFO: `/v3/users/bbcf2dc3345d4412a1a0edf92538b20b` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:06:56.354290 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ec8b36-27f5-4336-9c1f-0b9cfa023f64 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bbcf2dc3345d4412a1a0edf92538b20b)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:06:56.356300 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ec8b36-27f5-4336-9c1f-0b9cfa023f64 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:06:56.423049 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-69ec8b36-27f5-4336-9c1f-0b9cfa023f64 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bbcf2dc3345d4412a1a0edf92538b20b'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:06:56.431218 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-69ec8b36-27f5-4336-9c1f-0b9cfa023f64 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bbcf2dc3345d4412a1a0edf92538b20b'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:06:56.432018 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-69ec8b36-27f5-4336-9c1f-0b9cfa023f64 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bbcf2dc3345d4412a1a0edf92538b20b'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:06:56.446766 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-69ec8b36-27f5-4336-9c1f-0b9cfa023f64 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bbcf2dc3345d4412a1a0edf92538b20b'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:06:56.447637 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-69ec8b36-27f5-4336-9c1f-0b9cfa023f64 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bbcf2dc3345d4412a1a0edf92538b20b'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:06:56.462365 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1151/2318] 10.222.0.40 () {58 vars in 1224 bytes} [Fri Aug 26 11:06:56 2022] DELETE /identity/v3/users/bbcf2dc3345d4412a1a0edf92538b20b => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 11:06:56.466879 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1e89d0-0152-4fe4-a58f-49abcbf5c838 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:06:56.467387 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1e89d0-0152-4fe4-a58f-49abcbf5c838 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:06:56.467543 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1e89d0-0152-4fe4-a58f-49abcbf5c838 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:06:56.488724 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-db1e89d0-0152-4fe4-a58f-49abcbf5c838 None None] MFA Rules not processed for user `ca155a48e2024c32af3fd2e19e5fac11`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 11:06:56.490610 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-db1e89d0-0152-4fe4-a58f-49abcbf5c838 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:06:56.542431 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 11:06:56.561058 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1168/2319] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 11:06:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 11:06:56.572178 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ec8b36-27f5-4336-9c1f-0b9cfa023f64 admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:06:56.573668 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-69ec8b36-27f5-4336-9c1f-0b9cfa023f64 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:06:56.648066 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ec8b36-27f5-4336-9c1f-0b9cfa023f64 admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:06:56.651801 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0650bc-3c57-491b-ac14-b87ee2ebeb1f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91da2acd2534423091537e1ba068d2b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e30ccf0c1204727b786876c141dbcd2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:06:56.652413 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0650bc-3c57-491b-ac14-b87ee2ebeb1f service neutron] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:06:56.652743 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0650bc-3c57-491b-ac14-b87ee2ebeb1f service neutron] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:06:56.652993 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0650bc-3c57-491b-ac14-b87ee2ebeb1f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:06:56.664916 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d0650bc-3c57-491b-ac14-b87ee2ebeb1f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:06:56.666411 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d0650bc-3c57-491b-ac14-b87ee2ebeb1f service neutron] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:06:56.677257 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1152/2320] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 11:06:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 11:06:57.131714 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1e89d0-0152-4fe4-a58f-49abcbf5c838 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:06:57.133375 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-db1e89d0-0152-4fe4-a58f-49abcbf5c838 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:06:57.158609 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1e89d0-0152-4fe4-a58f-49abcbf5c838 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:06:57.165107 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b6c40a-7037-48d0-be2c-369203f08aae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:06:57.165909 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b6c40a-7037-48d0-be2c-369203f08aae admin admin] REQUEST_METHOD: `DELETE` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:06:57.166218 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b6c40a-7037-48d0-be2c-369203f08aae admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:06:57.166599 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b6c40a-7037-48d0-be2c-369203f08aae admin admin] PATH_INFO: `/v3/projects/0b56b31c31ba4121aa2271ca045eb40c` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:06:57.167787 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b6c40a-7037-48d0-be2c-369203f08aae admin admin] RBAC: Authorizing `identity:delete_project(project_id=0b56b31c31ba4121aa2271ca045eb40c)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:06:57.169853 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b6c40a-7037-48d0-be2c-369203f08aae admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:06:57.245868 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-a2b6c40a-7037-48d0-be2c-369203f08aae admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0b56b31c31ba4121aa2271ca045eb40c', 'request_id': 'req-a2b6c40a-7037-48d0-be2c-369203f08aae'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:06:57.253063 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-a2b6c40a-7037-48d0-be2c-369203f08aae admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0b56b31c31ba4121aa2271ca045eb40c', 'request_id': 'req-a2b6c40a-7037-48d0-be2c-369203f08aae'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:06:57.259665 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-a2b6c40a-7037-48d0-be2c-369203f08aae admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0b56b31c31ba4121aa2271ca045eb40c', 'request_id': 'req-a2b6c40a-7037-48d0-be2c-369203f08aae'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:06:57.278015 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-a2b6c40a-7037-48d0-be2c-369203f08aae admin admin] The token cache is being invalidate because project 0b56b31c31ba4121aa2271ca045eb40c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81424) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 11:06:57.278411 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-a2b6c40a-7037-48d0-be2c-369203f08aae admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:06:57.279168 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-a2b6c40a-7037-48d0-be2c-369203f08aae admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:06:57.285662 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1169/2321] 10.222.0.40 () {58 vars in 1233 bytes} [Fri Aug 26 11:06:57 2022] DELETE /identity/v3/projects/0b56b31c31ba4121aa2271ca045eb40c => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 11:06:57.286517 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0650bc-3c57-491b-ac14-b87ee2ebeb1f service neutron] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:06:57.302203 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0650bc-3c57-491b-ac14-b87ee2ebeb1f service neutron] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:06:57.374539 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 11:06:57.396598 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c3ccc7-6216-4ef9-848b-60b267df1e4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:06:57.397320 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c3ccc7-6216-4ef9-848b-60b267df1e4e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:06:57.397665 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c3ccc7-6216-4ef9-848b-60b267df1e4e admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:06:57.398018 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c3ccc7-6216-4ef9-848b-60b267df1e4e admin admin] PATH_INFO: `/v3/users/44bb554f1bfa4858a7f93c9a2e12cee3` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:06:57.399363 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c3ccc7-6216-4ef9-848b-60b267df1e4e admin admin] RBAC: Authorizing `identity:delete_user(user_id=44bb554f1bfa4858a7f93c9a2e12cee3)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:06:57.401347 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c3ccc7-6216-4ef9-848b-60b267df1e4e admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:06:57.473083 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-45c3ccc7-6216-4ef9-848b-60b267df1e4e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '44bb554f1bfa4858a7f93c9a2e12cee3'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:06:57.481958 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-45c3ccc7-6216-4ef9-848b-60b267df1e4e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '44bb554f1bfa4858a7f93c9a2e12cee3'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:06:57.482841 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-45c3ccc7-6216-4ef9-848b-60b267df1e4e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '44bb554f1bfa4858a7f93c9a2e12cee3'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:06:57.497814 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-45c3ccc7-6216-4ef9-848b-60b267df1e4e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '44bb554f1bfa4858a7f93c9a2e12cee3'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:06:57.498800 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-45c3ccc7-6216-4ef9-848b-60b267df1e4e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '44bb554f1bfa4858a7f93c9a2e12cee3'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:06:57.517599 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1153/2322] 10.222.0.40 () {58 vars in 1224 bytes} [Fri Aug 26 11:06:57 2022] DELETE /identity/v3/users/44bb554f1bfa4858a7f93c9a2e12cee3 => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 26 11:06:57.832527 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b6c40a-7037-48d0-be2c-369203f08aae admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:06:57.834960 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-a2b6c40a-7037-48d0-be2c-369203f08aae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:06:57.857286 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b6c40a-7037-48d0-be2c-369203f08aae admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:06:57.923234 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7fe1a2-e912-42bc-bf10-a3468f177e7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:06:57.923792 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7fe1a2-e912-42bc-bf10-a3468f177e7a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:06:57.924068 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7fe1a2-e912-42bc-bf10-a3468f177e7a admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:06:57.924342 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7fe1a2-e912-42bc-bf10-a3468f177e7a admin admin] PATH_INFO: `/v3/projects/6e49f1020e6a4b98b4882b0bae053da4` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:06:57.925485 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f7fe1a2-e912-42bc-bf10-a3468f177e7a admin admin] RBAC: Authorizing `identity:delete_project(project_id=6e49f1020e6a4b98b4882b0bae053da4)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:06:57.928081 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f7fe1a2-e912-42bc-bf10-a3468f177e7a admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:06:57.981938 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-0f7fe1a2-e912-42bc-bf10-a3468f177e7a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6e49f1020e6a4b98b4882b0bae053da4', 'request_id': 'req-0f7fe1a2-e912-42bc-bf10-a3468f177e7a'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:06:57.990011 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-0f7fe1a2-e912-42bc-bf10-a3468f177e7a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6e49f1020e6a4b98b4882b0bae053da4', 'request_id': 'req-0f7fe1a2-e912-42bc-bf10-a3468f177e7a'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:06:57.997547 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-0f7fe1a2-e912-42bc-bf10-a3468f177e7a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6e49f1020e6a4b98b4882b0bae053da4', 'request_id': 'req-0f7fe1a2-e912-42bc-bf10-a3468f177e7a'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:06:58.014699 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 11:06:58.023436 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-0f7fe1a2-e912-42bc-bf10-a3468f177e7a admin admin] The token cache is being invalidate because project 6e49f1020e6a4b98b4882b0bae053da4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81424) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 11:06:58.023819 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-0f7fe1a2-e912-42bc-bf10-a3468f177e7a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:06:58.024536 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-0f7fe1a2-e912-42bc-bf10-a3468f177e7a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:06:58.027966 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1170/2323] 10.222.0.40 () {58 vars in 1233 bytes} [Fri Aug 26 11:06:57 2022] DELETE /identity/v3/projects/6e49f1020e6a4b98b4882b0bae053da4 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 11:06:58.033021 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d303816-4d72-4aa2-afc4-65520d098387 None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:06:58.033345 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d303816-4d72-4aa2-afc4-65520d098387 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:06:58.033632 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d303816-4d72-4aa2-afc4-65520d098387 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:06:58.056010 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-5d303816-4d72-4aa2-afc4-65520d098387 None None] MFA Rules not processed for user `ca155a48e2024c32af3fd2e19e5fac11`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 11:06:58.057232 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-5d303816-4d72-4aa2-afc4-65520d098387 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:06:58.138124 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1154/2324] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 11:06:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 11:06:58.147033 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24cf674d-c132-4346-b15b-a6011274f1dc None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:06:58.147566 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24cf674d-c132-4346-b15b-a6011274f1dc None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:06:58.148070 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24cf674d-c132-4346-b15b-a6011274f1dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:06:58.171259 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-24cf674d-c132-4346-b15b-a6011274f1dc None None] MFA Rules not processed for user `ca155a48e2024c32af3fd2e19e5fac11`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 11:06:58.172845 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-24cf674d-c132-4346-b15b-a6011274f1dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:06:58.182390 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1171/2325] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 11:06:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 11:06:58.187097 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d303816-4d72-4aa2-afc4-65520d098387 None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:06:58.197703 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d303816-4d72-4aa2-afc4-65520d098387 None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:06:58.202616 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2733e015-29a9-4883-b5d5-1c5c945e6284 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:06:58.203228 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2733e015-29a9-4883-b5d5-1c5c945e6284 admin admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:06:58.203508 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2733e015-29a9-4883-b5d5-1c5c945e6284 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:06:58.203772 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2733e015-29a9-4883-b5d5-1c5c945e6284 admin admin] PATH_INFO: `/v3/domains` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:06:58.204429 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2733e015-29a9-4883-b5d5-1c5c945e6284 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 11:06:58.204726 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2733e015-29a9-4883-b5d5-1c5c945e6284 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:06:58.206520 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2733e015-29a9-4883-b5d5-1c5c945e6284 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:06:58.225914 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1155/2326] 10.222.0.40 () {58 vars in 1153 bytes} [Fri Aug 26 11:06:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 11:06:58.230221 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24cf674d-c132-4346-b15b-a6011274f1dc None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:06:58.243221 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24cf674d-c132-4346-b15b-a6011274f1dc None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:06:58.248091 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8578661d-b1f9-4a0e-a29f-7bc6b86b2607 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:06:58.248720 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8578661d-b1f9-4a0e-a29f-7bc6b86b2607 admin admin] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:06:58.248979 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8578661d-b1f9-4a0e-a29f-7bc6b86b2607 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:06:58.249427 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8578661d-b1f9-4a0e-a29f-7bc6b86b2607 admin admin] PATH_INFO: `/v3/projects` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:06:58.250096 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8578661d-b1f9-4a0e-a29f-7bc6b86b2607 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:06:58.252267 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8578661d-b1f9-4a0e-a29f-7bc6b86b2607 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:06:58.299694 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1172/2327] 10.222.0.40 () {60 vars in 1153 bytes} [Fri Aug 26 11:06:58 2022] POST /identity/v3/projects => generated 385 bytes in 70 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 26 11:06:58.304730 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2733e015-29a9-4883-b5d5-1c5c945e6284 admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:06:58.318227 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2733e015-29a9-4883-b5d5-1c5c945e6284 admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:06:58.389373 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 11:06:58.425598 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c0fe5b-e902-4cf4-b4d2-34032a0fc101 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:06:58.426302 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c0fe5b-e902-4cf4-b4d2-34032a0fc101 admin admin] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:06:58.426638 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c0fe5b-e902-4cf4-b4d2-34032a0fc101 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:06:58.426910 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c0fe5b-e902-4cf4-b4d2-34032a0fc101 admin admin] PATH_INFO: `/v3/users` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:06:58.427795 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c0fe5b-e902-4cf4-b4d2-34032a0fc101 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:06:58.430022 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c0fe5b-e902-4cf4-b4d2-34032a0fc101 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:06:58.468090 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1156/2328] 10.222.0.40 () {60 vars in 1144 bytes} [Fri Aug 26 11:06:58 2022] POST /identity/v3/users => generated 293 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 26 11:06:58.469476 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8578661d-b1f9-4a0e-a29f-7bc6b86b2607 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:06:58.481135 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8578661d-b1f9-4a0e-a29f-7bc6b86b2607 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:06:58.485937 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37be3aa0-a829-41ae-af7a-6ebd658bfd09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:06:58.486609 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37be3aa0-a829-41ae-af7a-6ebd658bfd09 admin admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:06:58.486889 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37be3aa0-a829-41ae-af7a-6ebd658bfd09 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:06:58.487407 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37be3aa0-a829-41ae-af7a-6ebd658bfd09 admin admin] PATH_INFO: `/v3/roles` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:06:58.488079 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37be3aa0-a829-41ae-af7a-6ebd658bfd09 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:06:58.490437 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37be3aa0-a829-41ae-af7a-6ebd658bfd09 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:06:58.505563 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1173/2329] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 26 11:06:58 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 11:06:58.506834 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c0fe5b-e902-4cf4-b4d2-34032a0fc101 admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:06:58.521305 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c0fe5b-e902-4cf4-b4d2-34032a0fc101 admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:06:58.527368 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d152912a-797c-437c-82fd-f8c87d38f844 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:06:58.528121 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d152912a-797c-437c-82fd-f8c87d38f844 admin admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:06:58.528408 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d152912a-797c-437c-82fd-f8c87d38f844 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:06:58.528683 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d152912a-797c-437c-82fd-f8c87d38f844 admin admin] PATH_INFO: `/v3/roles` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:06:58.529342 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d152912a-797c-437c-82fd-f8c87d38f844 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:06:58.530992 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d152912a-797c-437c-82fd-f8c87d38f844 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:06:58.545178 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1157/2330] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 26 11:06:58 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 11:06:58.547835 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37be3aa0-a829-41ae-af7a-6ebd658bfd09 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:06:58.558587 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37be3aa0-a829-41ae-af7a-6ebd658bfd09 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:06:58.564056 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4046a2d6-0e90-42e4-bce9-3de7c8804d29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:06:58.564787 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4046a2d6-0e90-42e4-bce9-3de7c8804d29 admin admin] REQUEST_METHOD: `PUT` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:06:58.565073 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4046a2d6-0e90-42e4-bce9-3de7c8804d29 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:06:58.565352 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4046a2d6-0e90-42e4-bce9-3de7c8804d29 admin admin] PATH_INFO: `/v3/projects/7c8b6f1e1fbc4cdb9d7271e3560b55a2/users/8ca5d206651748f9a911ecd5521d8b3b/roles/23e174ce743b46eabe618c212691da55` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:06:58.586686 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4046a2d6-0e90-42e4-bce9-3de7c8804d29 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7c8b6f1e1fbc4cdb9d7271e3560b55a2, user_id=8ca5d206651748f9a911ecd5521d8b3b, role_id=23e174ce743b46eabe618c212691da55)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:06:58.589456 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4046a2d6-0e90-42e4-bce9-3de7c8804d29 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:06:58.604854 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1174/2331] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 26 11:06:58 2022] PUT /identity/v3/projects/7c8b6f1e1fbc4cdb9d7271e3560b55a2/users/8ca5d206651748f9a911ecd5521d8b3b/roles/23e174ce743b46eabe618c212691da55 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 11:06:58.616366 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f63f0f-3953-4eef-b583-a47a73993327 None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:06:58.616670 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f63f0f-3953-4eef-b583-a47a73993327 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:06:58.617086 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f63f0f-3953-4eef-b583-a47a73993327 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:06:58.654539 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-68f63f0f-3953-4eef-b583-a47a73993327 None None] MFA Rules not processed for user `8ca5d206651748f9a911ecd5521d8b3b`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 11:06:58.655831 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-68f63f0f-3953-4eef-b583-a47a73993327 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:06:58.739593 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1158/2332] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 11:06:58 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 11:06:58.746823 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4046a2d6-0e90-42e4-bce9-3de7c8804d29 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:06:58.817751 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4046a2d6-0e90-42e4-bce9-3de7c8804d29 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:06:58.822888 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32bb4072-4136-4bff-8d0e-fe75a1330c9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:06:58.823513 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32bb4072-4136-4bff-8d0e-fe75a1330c9a admin admin] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:06:58.824285 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32bb4072-4136-4bff-8d0e-fe75a1330c9a admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:06:58.824285 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32bb4072-4136-4bff-8d0e-fe75a1330c9a admin admin] PATH_INFO: `/v3/projects` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:06:58.824697 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32bb4072-4136-4bff-8d0e-fe75a1330c9a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:06:58.826724 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32bb4072-4136-4bff-8d0e-fe75a1330c9a admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:06:58.859851 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1175/2333] 10.222.0.40 () {60 vars in 1153 bytes} [Fri Aug 26 11:06:58 2022] POST /identity/v3/projects => generated 385 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 11:06:58.861407 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f63f0f-3953-4eef-b583-a47a73993327 None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:06:58.874754 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f63f0f-3953-4eef-b583-a47a73993327 None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:06:58.947528 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061a003f-333d-4f82-8a66-cb6651ad8276 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:06:58.948555 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061a003f-333d-4f82-8a66-cb6651ad8276 admin admin] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:06:58.948847 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061a003f-333d-4f82-8a66-cb6651ad8276 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:06:58.949242 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061a003f-333d-4f82-8a66-cb6651ad8276 admin admin] PATH_INFO: `/v3/users` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:06:58.950477 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-061a003f-333d-4f82-8a66-cb6651ad8276 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:06:58.953347 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-061a003f-333d-4f82-8a66-cb6651ad8276 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:06:58.983418 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1159/2334] 10.222.0.40 () {60 vars in 1144 bytes} [Fri Aug 26 11:06:58 2022] POST /identity/v3/users => generated 299 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 11:06:58.988517 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32bb4072-4136-4bff-8d0e-fe75a1330c9a admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:06:59.002643 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32bb4072-4136-4bff-8d0e-fe75a1330c9a admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:06:59.010120 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a9788c-af98-4937-80a2-a9da91a84fed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:06:59.011156 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a9788c-af98-4937-80a2-a9da91a84fed admin admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:06:59.011443 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a9788c-af98-4937-80a2-a9da91a84fed admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:06:59.011720 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a9788c-af98-4937-80a2-a9da91a84fed admin admin] PATH_INFO: `/v3/roles` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:06:59.012748 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a9788c-af98-4937-80a2-a9da91a84fed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:06:59.014779 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a9788c-af98-4937-80a2-a9da91a84fed admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:06:59.028385 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1176/2335] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 26 11:06:58 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 11:06:59.032728 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061a003f-333d-4f82-8a66-cb6651ad8276 admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:06:59.048890 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061a003f-333d-4f82-8a66-cb6651ad8276 admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:06:59.056515 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65029ac2-6f6f-4df5-9302-c523e453907d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:06:59.057191 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65029ac2-6f6f-4df5-9302-c523e453907d admin admin] REQUEST_METHOD: `PUT` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:06:59.057465 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65029ac2-6f6f-4df5-9302-c523e453907d admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:06:59.057891 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65029ac2-6f6f-4df5-9302-c523e453907d admin admin] PATH_INFO: `/v3/projects/d01e5fa84594470188b9ea9f8596ed49/users/2e59fdb395da4d10980308329616f24c/roles/1f97e4602e2e4d6498913a81f9ae8d6a` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:06:59.078269 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65029ac2-6f6f-4df5-9302-c523e453907d admin admin] RBAC: Authorizing `identity:create_grant(project_id=d01e5fa84594470188b9ea9f8596ed49, user_id=2e59fdb395da4d10980308329616f24c, role_id=1f97e4602e2e4d6498913a81f9ae8d6a)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:06:59.083206 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65029ac2-6f6f-4df5-9302-c523e453907d admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:06:59.101526 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1160/2336] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 26 11:06:59 2022] PUT /identity/v3/projects/d01e5fa84594470188b9ea9f8596ed49/users/2e59fdb395da4d10980308329616f24c/roles/1f97e4602e2e4d6498913a81f9ae8d6a => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 11:06:59.105964 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a9788c-af98-4937-80a2-a9da91a84fed admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:06:59.119422 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a9788c-af98-4937-80a2-a9da91a84fed admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:06:59.186857 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2b8be6-b459-48d1-bfaa-00e28525b8b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:06:59.187477 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2b8be6-b459-48d1-bfaa-00e28525b8b6 admin admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:06:59.187754 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2b8be6-b459-48d1-bfaa-00e28525b8b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:06:59.188026 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2b8be6-b459-48d1-bfaa-00e28525b8b6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:06:59.188704 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b2b8be6-b459-48d1-bfaa-00e28525b8b6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:06:59.190384 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b2b8be6-b459-48d1-bfaa-00e28525b8b6 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:06:59.201490 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1177/2337] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 26 11:06:59 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 11:06:59.205704 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65029ac2-6f6f-4df5-9302-c523e453907d admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:06:59.218634 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65029ac2-6f6f-4df5-9302-c523e453907d admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:06:59.224641 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1011e963-3d82-47ca-9e67-21fdf66d38c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:06:59.224641 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1011e963-3d82-47ca-9e67-21fdf66d38c3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:06:59.224641 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1011e963-3d82-47ca-9e67-21fdf66d38c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:06:59.224641 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1011e963-3d82-47ca-9e67-21fdf66d38c3 admin admin] PATH_INFO: `/v3/projects/d01e5fa84594470188b9ea9f8596ed49/users/2e59fdb395da4d10980308329616f24c/roles/1f97e4602e2e4d6498913a81f9ae8d6a` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:06:59.228616 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1011e963-3d82-47ca-9e67-21fdf66d38c3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d01e5fa84594470188b9ea9f8596ed49, user_id=2e59fdb395da4d10980308329616f24c, role_id=1f97e4602e2e4d6498913a81f9ae8d6a)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:06:59.231306 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1011e963-3d82-47ca-9e67-21fdf66d38c3 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:06:59.263944 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1161/2338] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 26 11:06:59 2022] PUT /identity/v3/projects/d01e5fa84594470188b9ea9f8596ed49/users/2e59fdb395da4d10980308329616f24c/roles/1f97e4602e2e4d6498913a81f9ae8d6a => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 11:06:59.274477 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff3df1c-0255-49c7-986c-493a073c37e4 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:06:59.274760 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff3df1c-0255-49c7-986c-493a073c37e4 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:06:59.275005 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff3df1c-0255-49c7-986c-493a073c37e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:06:59.291201 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 11:06:59.312028 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-aff3df1c-0255-49c7-986c-493a073c37e4 None None] MFA Rules not processed for user `2e59fdb395da4d10980308329616f24c`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 11:06:59.313573 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-aff3df1c-0255-49c7-986c-493a073c37e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:06:59.418443 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1178/2339] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 11:06:59 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 11:06:59.433481 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1011e963-3d82-47ca-9e67-21fdf66d38c3 admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:06:59.437022 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-1011e963-3d82-47ca-9e67-21fdf66d38c3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:06:59.467646 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 11:06:59.533413 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1011e963-3d82-47ca-9e67-21fdf66d38c3 admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:06:59.542607 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88669a20-0b5c-4333-b72f-721ba05e1cd6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c277338a7f814cc6b01c5419775031bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e30ccf0c1204727b786876c141dbcd2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:06:59.543357 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88669a20-0b5c-4333-b72f-721ba05e1cd6 service cinder] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:06:59.543676 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88669a20-0b5c-4333-b72f-721ba05e1cd6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:06:59.544159 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88669a20-0b5c-4333-b72f-721ba05e1cd6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:06:59.687900 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88669a20-0b5c-4333-b72f-721ba05e1cd6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:06:59.693998 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88669a20-0b5c-4333-b72f-721ba05e1cd6 service cinder] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:06:59.703127 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1162/2340] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 11:06:59 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 270 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 11:07:04.208909 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff3df1c-0255-49c7-986c-493a073c37e4 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:07:04.210409 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-aff3df1c-0255-49c7-986c-493a073c37e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:07:04.222128 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff3df1c-0255-49c7-986c-493a073c37e4 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:07:04.226422 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be63a3b4-e966-40b2-9522-c65133db7b9f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c277338a7f814cc6b01c5419775031bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e30ccf0c1204727b786876c141dbcd2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:07:04.227029 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be63a3b4-e966-40b2-9522-c65133db7b9f service cinder] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:07:04.227301 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be63a3b4-e966-40b2-9522-c65133db7b9f service cinder] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:07:04.227571 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be63a3b4-e966-40b2-9522-c65133db7b9f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:07:04.236841 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be63a3b4-e966-40b2-9522-c65133db7b9f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:07:04.238296 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be63a3b4-e966-40b2-9522-c65133db7b9f service cinder] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:07:04.248934 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1179/2341] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 11:07:04 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 11:07:08.776702 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a04c485-5049-4079-9057-f7bd44260e6a None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:07:08.777268 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a04c485-5049-4079-9057-f7bd44260e6a None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:07:08.777268 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a04c485-5049-4079-9057-f7bd44260e6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:07:08.799691 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-5a04c485-5049-4079-9057-f7bd44260e6a None None] MFA Rules not processed for user `50be12ccbd274a8792615e9dd10b61ca`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 11:07:08.800926 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-5a04c485-5049-4079-9057-f7bd44260e6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:07:08.921847 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be63a3b4-e966-40b2-9522-c65133db7b9f service cinder] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:07:08.923307 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-be63a3b4-e966-40b2-9522-c65133db7b9f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:07:08.932856 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1163/2342] 10.222.0.40 () {58 vars in 1009 bytes} [Fri Aug 26 11:07:08 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 11:07:09.025308 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be63a3b4-e966-40b2-9522-c65133db7b9f service cinder] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:07:09.036971 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afce2145-642d-4ad0-8361-ea98536080ab service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6529bea296c9482a973ced5fe9f2212f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e30ccf0c1204727b786876c141dbcd2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:07:09.037785 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afce2145-642d-4ad0-8361-ea98536080ab service swift] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:07:09.038006 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afce2145-642d-4ad0-8361-ea98536080ab service swift] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:07:09.038349 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afce2145-642d-4ad0-8361-ea98536080ab service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:07:09.069595 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afce2145-642d-4ad0-8361-ea98536080ab service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:07:09.072566 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afce2145-642d-4ad0-8361-ea98536080ab service swift] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:07:09.085341 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1180/2343] 10.222.0.40 () {60 vars in 1384 bytes} [Fri Aug 26 11:07:08 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 164 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 26 11:07:09.523761 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a04c485-5049-4079-9057-f7bd44260e6a None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:07:09.672550 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a04c485-5049-4079-9057-f7bd44260e6a None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:07:09.679786 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459187c9-ba76-406f-800a-36ec77fcd7dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:07:09.680583 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459187c9-ba76-406f-800a-36ec77fcd7dc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:07:09.681035 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459187c9-ba76-406f-800a-36ec77fcd7dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:07:09.681413 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459187c9-ba76-406f-800a-36ec77fcd7dc admin admin] PATH_INFO: `/v3/users/8ca5d206651748f9a911ecd5521d8b3b` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:07:09.682792 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-459187c9-ba76-406f-800a-36ec77fcd7dc admin admin] RBAC: Authorizing `identity:delete_user(user_id=8ca5d206651748f9a911ecd5521d8b3b)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:07:09.684854 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-459187c9-ba76-406f-800a-36ec77fcd7dc admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:07:09.710174 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 11:07:09.789239 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-459187c9-ba76-406f-800a-36ec77fcd7dc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8ca5d206651748f9a911ecd5521d8b3b'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:07:09.802209 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-459187c9-ba76-406f-800a-36ec77fcd7dc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8ca5d206651748f9a911ecd5521d8b3b'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:07:09.803871 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-459187c9-ba76-406f-800a-36ec77fcd7dc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8ca5d206651748f9a911ecd5521d8b3b'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:07:09.831624 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-459187c9-ba76-406f-800a-36ec77fcd7dc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8ca5d206651748f9a911ecd5521d8b3b'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:07:09.833221 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-459187c9-ba76-406f-800a-36ec77fcd7dc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8ca5d206651748f9a911ecd5521d8b3b'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:07:09.866541 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1164/2344] 10.222.0.40 () {58 vars in 1224 bytes} [Fri Aug 26 11:07:09 2022] DELETE /identity/v3/users/8ca5d206651748f9a911ecd5521d8b3b => generated 0 bytes in 340 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 11:07:09.869086 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890cef57-e4fc-4f7d-93f2-c714759d3298 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:07:09.869371 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890cef57-e4fc-4f7d-93f2-c714759d3298 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:07:09.869643 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890cef57-e4fc-4f7d-93f2-c714759d3298 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:07:09.943047 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-890cef57-e4fc-4f7d-93f2-c714759d3298 None None] MFA Rules not processed for user `ca155a48e2024c32af3fd2e19e5fac11`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 11:07:09.946168 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-890cef57-e4fc-4f7d-93f2-c714759d3298 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:07:10.131591 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1181/2345] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 11:07:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 264 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 11:07:10.144157 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459187c9-ba76-406f-800a-36ec77fcd7dc admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:07:10.148878 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-459187c9-ba76-406f-800a-36ec77fcd7dc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:07:10.267365 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459187c9-ba76-406f-800a-36ec77fcd7dc admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:07:10.279344 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75da5590-2181-4203-9dd9-1eddbd0bd926 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91da2acd2534423091537e1ba068d2b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e30ccf0c1204727b786876c141dbcd2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:07:10.281091 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75da5590-2181-4203-9dd9-1eddbd0bd926 service neutron] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:07:10.281893 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75da5590-2181-4203-9dd9-1eddbd0bd926 service neutron] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:07:10.282764 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75da5590-2181-4203-9dd9-1eddbd0bd926 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:07:10.301502 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75da5590-2181-4203-9dd9-1eddbd0bd926 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:07:10.303969 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75da5590-2181-4203-9dd9-1eddbd0bd926 service neutron] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:07:10.320683 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1165/2346] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 11:07:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 11:07:10.831434 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890cef57-e4fc-4f7d-93f2-c714759d3298 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:07:10.833182 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-890cef57-e4fc-4f7d-93f2-c714759d3298 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:07:10.861487 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890cef57-e4fc-4f7d-93f2-c714759d3298 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:07:10.867963 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e839cf1-6e69-4c96-92af-f0b1a200842f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:07:10.868968 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e839cf1-6e69-4c96-92af-f0b1a200842f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:07:10.870238 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e839cf1-6e69-4c96-92af-f0b1a200842f admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:07:10.871273 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e839cf1-6e69-4c96-92af-f0b1a200842f admin admin] PATH_INFO: `/v3/projects/7c8b6f1e1fbc4cdb9d7271e3560b55a2` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:07:10.875550 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e839cf1-6e69-4c96-92af-f0b1a200842f admin admin] RBAC: Authorizing `identity:delete_project(project_id=7c8b6f1e1fbc4cdb9d7271e3560b55a2)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:07:10.878449 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e839cf1-6e69-4c96-92af-f0b1a200842f admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:07:10.885372 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 11:07:10.960793 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-4e839cf1-6e69-4c96-92af-f0b1a200842f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7c8b6f1e1fbc4cdb9d7271e3560b55a2', 'request_id': 'req-4e839cf1-6e69-4c96-92af-f0b1a200842f'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:07:10.971183 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-4e839cf1-6e69-4c96-92af-f0b1a200842f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7c8b6f1e1fbc4cdb9d7271e3560b55a2', 'request_id': 'req-4e839cf1-6e69-4c96-92af-f0b1a200842f'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:07:10.986126 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-4e839cf1-6e69-4c96-92af-f0b1a200842f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7c8b6f1e1fbc4cdb9d7271e3560b55a2', 'request_id': 'req-4e839cf1-6e69-4c96-92af-f0b1a200842f'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:07:11.009931 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-4e839cf1-6e69-4c96-92af-f0b1a200842f admin admin] The token cache is being invalidate because project 7c8b6f1e1fbc4cdb9d7271e3560b55a2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81424) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 11:07:11.011413 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-4e839cf1-6e69-4c96-92af-f0b1a200842f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:07:11.012348 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-4e839cf1-6e69-4c96-92af-f0b1a200842f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:07:11.018541 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1182/2347] 10.222.0.40 () {58 vars in 1233 bytes} [Fri Aug 26 11:07:10 2022] DELETE /identity/v3/projects/7c8b6f1e1fbc4cdb9d7271e3560b55a2 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 11:07:11.019038 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75da5590-2181-4203-9dd9-1eddbd0bd926 service neutron] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:07:11.034364 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75da5590-2181-4203-9dd9-1eddbd0bd926 service neutron] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:07:11.202609 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8143d6f9-21e3-42b8-ba58-57743caa303e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:07:11.202609 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8143d6f9-21e3-42b8-ba58-57743caa303e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:07:11.202609 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8143d6f9-21e3-42b8-ba58-57743caa303e admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:07:11.203110 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8143d6f9-21e3-42b8-ba58-57743caa303e admin admin] PATH_INFO: `/v3/users/2e59fdb395da4d10980308329616f24c` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:07:11.205576 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8143d6f9-21e3-42b8-ba58-57743caa303e admin admin] RBAC: Authorizing `identity:delete_user(user_id=2e59fdb395da4d10980308329616f24c)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:07:11.208335 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8143d6f9-21e3-42b8-ba58-57743caa303e admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:07:11.293397 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:DELETE=15,keystone:SELECT=84,keystone:INSERT=1 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 11:07:11.322255 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-8143d6f9-21e3-42b8-ba58-57743caa303e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2e59fdb395da4d10980308329616f24c'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:07:11.338558 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-8143d6f9-21e3-42b8-ba58-57743caa303e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2e59fdb395da4d10980308329616f24c'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:07:11.339673 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-8143d6f9-21e3-42b8-ba58-57743caa303e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2e59fdb395da4d10980308329616f24c'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:07:11.363460 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-8143d6f9-21e3-42b8-ba58-57743caa303e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2e59fdb395da4d10980308329616f24c'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:07:11.365120 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-8143d6f9-21e3-42b8-ba58-57743caa303e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2e59fdb395da4d10980308329616f24c'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:07:11.398621 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1166/2348] 10.222.0.40 () {58 vars in 1224 bytes} [Fri Aug 26 11:07:11 2022] DELETE /identity/v3/users/2e59fdb395da4d10980308329616f24c => generated 0 bytes in 380 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 26 11:07:12.311076 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e839cf1-6e69-4c96-92af-f0b1a200842f admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:07:12.312905 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-4e839cf1-6e69-4c96-92af-f0b1a200842f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:07:12.344202 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e839cf1-6e69-4c96-92af-f0b1a200842f admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:07:12.463080 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095f401a-20c2-44c0-8bbc-ebce4e86cc01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:07:12.463728 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095f401a-20c2-44c0-8bbc-ebce4e86cc01 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:07:12.464168 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095f401a-20c2-44c0-8bbc-ebce4e86cc01 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:07:12.464652 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095f401a-20c2-44c0-8bbc-ebce4e86cc01 admin admin] PATH_INFO: `/v3/projects/d01e5fa84594470188b9ea9f8596ed49` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:07:12.465606 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-095f401a-20c2-44c0-8bbc-ebce4e86cc01 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d01e5fa84594470188b9ea9f8596ed49)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:07:12.468038 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-095f401a-20c2-44c0-8bbc-ebce4e86cc01 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:07:12.550964 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-095f401a-20c2-44c0-8bbc-ebce4e86cc01 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd01e5fa84594470188b9ea9f8596ed49', 'request_id': 'req-095f401a-20c2-44c0-8bbc-ebce4e86cc01'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:07:12.559198 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-095f401a-20c2-44c0-8bbc-ebce4e86cc01 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd01e5fa84594470188b9ea9f8596ed49', 'request_id': 'req-095f401a-20c2-44c0-8bbc-ebce4e86cc01'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:07:12.566355 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-095f401a-20c2-44c0-8bbc-ebce4e86cc01 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd01e5fa84594470188b9ea9f8596ed49', 'request_id': 'req-095f401a-20c2-44c0-8bbc-ebce4e86cc01'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:07:12.588124 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-095f401a-20c2-44c0-8bbc-ebce4e86cc01 admin admin] The token cache is being invalidate because project d01e5fa84594470188b9ea9f8596ed49 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81424) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 11:07:12.588236 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-095f401a-20c2-44c0-8bbc-ebce4e86cc01 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:07:12.589143 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-095f401a-20c2-44c0-8bbc-ebce4e86cc01 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:07:12.593549 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1183/2349] 10.222.0.40 () {58 vars in 1233 bytes} [Fri Aug 26 11:07:12 2022] DELETE /identity/v3/projects/d01e5fa84594470188b9ea9f8596ed49 => generated 0 bytes in 283 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 11:07:12.600855 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2e68d7-ce5e-4391-a504-6e3bf9f3ef63 None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:07:12.601379 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2e68d7-ce5e-4391-a504-6e3bf9f3ef63 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:07:12.601489 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2e68d7-ce5e-4391-a504-6e3bf9f3ef63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:07:12.631701 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-fe2e68d7-ce5e-4391-a504-6e3bf9f3ef63 None None] MFA Rules not processed for user `ca155a48e2024c32af3fd2e19e5fac11`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 11:07:12.633118 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-fe2e68d7-ce5e-4391-a504-6e3bf9f3ef63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:07:12.749565 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1167/2350] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 11:07:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 11:07:12.763924 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4f84c4-5e87-47d5-b972-4627522cdee1 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:07:12.764237 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4f84c4-5e87-47d5-b972-4627522cdee1 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:07:12.764528 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4f84c4-5e87-47d5-b972-4627522cdee1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:07:12.809618 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-bc4f84c4-5e87-47d5-b972-4627522cdee1 None None] MFA Rules not processed for user `ca155a48e2024c32af3fd2e19e5fac11`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 11:07:12.810802 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-bc4f84c4-5e87-47d5-b972-4627522cdee1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:07:12.836912 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1184/2351] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 11:07:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 11:07:12.841311 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2e68d7-ce5e-4391-a504-6e3bf9f3ef63 None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:07:12.862979 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2e68d7-ce5e-4391-a504-6e3bf9f3ef63 None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:07:12.873799 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbce3ffa-2925-4b8e-b3f5-a6dc496b8b48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:07:12.874649 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbce3ffa-2925-4b8e-b3f5-a6dc496b8b48 admin admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:07:12.874890 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbce3ffa-2925-4b8e-b3f5-a6dc496b8b48 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:07:12.875155 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbce3ffa-2925-4b8e-b3f5-a6dc496b8b48 admin admin] PATH_INFO: `/v3/domains` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:07:12.875814 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbce3ffa-2925-4b8e-b3f5-a6dc496b8b48 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 11:07:12.876114 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbce3ffa-2925-4b8e-b3f5-a6dc496b8b48 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:07:12.877720 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbce3ffa-2925-4b8e-b3f5-a6dc496b8b48 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:07:12.908945 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1168/2352] 10.222.0.40 () {58 vars in 1153 bytes} [Fri Aug 26 11:07:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 11:07:12.914012 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4f84c4-5e87-47d5-b972-4627522cdee1 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:07:12.929782 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4f84c4-5e87-47d5-b972-4627522cdee1 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:07:12.939758 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee0e4ac-e464-499d-abe0-c98a2fa6bc62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:07:12.940760 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee0e4ac-e464-499d-abe0-c98a2fa6bc62 admin admin] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:07:12.941453 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee0e4ac-e464-499d-abe0-c98a2fa6bc62 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:07:12.942684 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee0e4ac-e464-499d-abe0-c98a2fa6bc62 admin admin] PATH_INFO: `/v3/projects` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:07:12.943360 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee0e4ac-e464-499d-abe0-c98a2fa6bc62 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:07:12.945307 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee0e4ac-e464-499d-abe0-c98a2fa6bc62 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:07:13.010624 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 11:07:13.047922 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1185/2353] 10.222.0.40 () {60 vars in 1153 bytes} [Fri Aug 26 11:07:12 2022] POST /identity/v3/projects => generated 375 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 11:07:13.052990 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbce3ffa-2925-4b8e-b3f5-a6dc496b8b48 admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:07:13.064610 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbce3ffa-2925-4b8e-b3f5-a6dc496b8b48 admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:07:13.141753 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90304d54-86ec-45bf-af3f-08b72eba4305 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:07:13.142882 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90304d54-86ec-45bf-af3f-08b72eba4305 admin admin] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:07:13.143388 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90304d54-86ec-45bf-af3f-08b72eba4305 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:07:13.143869 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90304d54-86ec-45bf-af3f-08b72eba4305 admin admin] PATH_INFO: `/v3/users` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:07:13.145058 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90304d54-86ec-45bf-af3f-08b72eba4305 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:07:13.149687 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90304d54-86ec-45bf-af3f-08b72eba4305 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:07:13.184870 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1169/2354] 10.222.0.40 () {60 vars in 1144 bytes} [Fri Aug 26 11:07:13 2022] POST /identity/v3/users => generated 288 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 26 11:07:13.188897 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee0e4ac-e464-499d-abe0-c98a2fa6bc62 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:07:13.203045 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee0e4ac-e464-499d-abe0-c98a2fa6bc62 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:07:13.211080 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6a7cb7-05bc-44d3-ba9c-7550dc2cf024 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:07:13.211777 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6a7cb7-05bc-44d3-ba9c-7550dc2cf024 admin admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:07:13.212197 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6a7cb7-05bc-44d3-ba9c-7550dc2cf024 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:07:13.212471 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6a7cb7-05bc-44d3-ba9c-7550dc2cf024 admin admin] PATH_INFO: `/v3/roles` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:07:13.213148 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c6a7cb7-05bc-44d3-ba9c-7550dc2cf024 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:07:13.215100 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c6a7cb7-05bc-44d3-ba9c-7550dc2cf024 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:07:13.230861 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1186/2355] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 26 11:07:13 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 11:07:13.238322 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90304d54-86ec-45bf-af3f-08b72eba4305 admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:07:13.253443 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90304d54-86ec-45bf-af3f-08b72eba4305 admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:07:13.285328 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea993189-1c42-4b2e-99aa-a20a91175b13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:07:13.287023 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea993189-1c42-4b2e-99aa-a20a91175b13 admin admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:07:13.287569 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea993189-1c42-4b2e-99aa-a20a91175b13 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:07:13.288246 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea993189-1c42-4b2e-99aa-a20a91175b13 admin admin] PATH_INFO: `/v3/roles` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:07:13.289106 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea993189-1c42-4b2e-99aa-a20a91175b13 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:07:13.290999 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea993189-1c42-4b2e-99aa-a20a91175b13 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:07:13.309909 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1170/2356] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 26 11:07:13 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 11:07:13.314422 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6a7cb7-05bc-44d3-ba9c-7550dc2cf024 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:07:13.352689 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6a7cb7-05bc-44d3-ba9c-7550dc2cf024 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:07:13.375181 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d38557-d5a7-4157-b772-2362211c74e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:07:13.376030 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d38557-d5a7-4157-b772-2362211c74e9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:07:13.376405 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d38557-d5a7-4157-b772-2362211c74e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:07:13.376773 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d38557-d5a7-4157-b772-2362211c74e9 admin admin] PATH_INFO: `/v3/projects/7182450761da4e99a374be8630789d72/users/37c7cab523864c9796690f4b20ce95ac/roles/23e174ce743b46eabe618c212691da55` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:07:13.419802 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d38557-d5a7-4157-b772-2362211c74e9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7182450761da4e99a374be8630789d72, user_id=37c7cab523864c9796690f4b20ce95ac, role_id=23e174ce743b46eabe618c212691da55)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:07:13.423273 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d38557-d5a7-4157-b772-2362211c74e9 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:07:13.478304 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1187/2357] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 26 11:07:13 2022] PUT /identity/v3/projects/7182450761da4e99a374be8630789d72/users/37c7cab523864c9796690f4b20ce95ac/roles/23e174ce743b46eabe618c212691da55 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 26 11:07:13.483314 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d948571-2dda-4dde-ad48-50530ed56351 None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:07:13.483603 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d948571-2dda-4dde-ad48-50530ed56351 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:07:13.484208 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d948571-2dda-4dde-ad48-50530ed56351 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:07:13.528769 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-8d948571-2dda-4dde-ad48-50530ed56351 None None] MFA Rules not processed for user `37c7cab523864c9796690f4b20ce95ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 11:07:13.533209 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-8d948571-2dda-4dde-ad48-50530ed56351 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:07:13.547240 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 11:07:13.767333 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1171/2358] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 11:07:13 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 285 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 11:07:13.772361 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d38557-d5a7-4157-b772-2362211c74e9 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:07:13.862770 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d38557-d5a7-4157-b772-2362211c74e9 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:07:13.882474 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3265d47-7bd5-46e8-9812-7491893e7daa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:07:13.883506 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3265d47-7bd5-46e8-9812-7491893e7daa admin admin] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:07:13.883899 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3265d47-7bd5-46e8-9812-7491893e7daa admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:07:13.884626 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3265d47-7bd5-46e8-9812-7491893e7daa admin admin] PATH_INFO: `/v3/projects` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:07:13.885306 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3265d47-7bd5-46e8-9812-7491893e7daa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:07:13.887359 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3265d47-7bd5-46e8-9812-7491893e7daa admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:07:13.928833 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d948571-2dda-4dde-ad48-50530ed56351 None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:07:13.929784 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1188/2359] 10.222.0.40 () {60 vars in 1153 bytes} [Fri Aug 26 11:07:13 2022] POST /identity/v3/projects => generated 375 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 11:07:13.941369 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d948571-2dda-4dde-ad48-50530ed56351 None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:07:14.024879 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859bff58-2418-435a-835f-4645d87d65c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:07:14.025513 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859bff58-2418-435a-835f-4645d87d65c5 admin admin] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:07:14.025796 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859bff58-2418-435a-835f-4645d87d65c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:07:14.026103 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859bff58-2418-435a-835f-4645d87d65c5 admin admin] PATH_INFO: `/v3/users` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:07:14.026745 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-859bff58-2418-435a-835f-4645d87d65c5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:07:14.028746 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-859bff58-2418-435a-835f-4645d87d65c5 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:07:14.058009 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1172/2360] 10.222.0.40 () {60 vars in 1144 bytes} [Fri Aug 26 11:07:13 2022] POST /identity/v3/users => generated 294 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 11:07:14.059603 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3265d47-7bd5-46e8-9812-7491893e7daa admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:07:14.071474 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3265d47-7bd5-46e8-9812-7491893e7daa admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:07:14.114786 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bceec9f5-a7b2-46fa-af68-327662a2a040 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:07:14.115846 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bceec9f5-a7b2-46fa-af68-327662a2a040 admin admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:07:14.116554 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bceec9f5-a7b2-46fa-af68-327662a2a040 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:07:14.117041 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bceec9f5-a7b2-46fa-af68-327662a2a040 admin admin] PATH_INFO: `/v3/roles` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:07:14.118067 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bceec9f5-a7b2-46fa-af68-327662a2a040 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:07:14.120192 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bceec9f5-a7b2-46fa-af68-327662a2a040 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:07:14.137337 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1189/2361] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 26 11:07:14 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 26 11:07:14.160473 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859bff58-2418-435a-835f-4645d87d65c5 admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:07:14.183936 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859bff58-2418-435a-835f-4645d87d65c5 admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:07:14.188936 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eadaa2a5-c7e7-46cf-9785-928ddd49e6ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:07:14.189603 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eadaa2a5-c7e7-46cf-9785-928ddd49e6ea admin admin] REQUEST_METHOD: `PUT` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:07:14.189880 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eadaa2a5-c7e7-46cf-9785-928ddd49e6ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:07:14.190179 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eadaa2a5-c7e7-46cf-9785-928ddd49e6ea admin admin] PATH_INFO: `/v3/projects/435c67c00aae4e47892622bfde5f2db3/users/0ae4787c81e944ad9c9197ae8eec26b0/roles/1f97e4602e2e4d6498913a81f9ae8d6a` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:07:14.207016 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eadaa2a5-c7e7-46cf-9785-928ddd49e6ea admin admin] RBAC: Authorizing `identity:create_grant(project_id=435c67c00aae4e47892622bfde5f2db3, user_id=0ae4787c81e944ad9c9197ae8eec26b0, role_id=1f97e4602e2e4d6498913a81f9ae8d6a)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:07:14.210792 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eadaa2a5-c7e7-46cf-9785-928ddd49e6ea admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:07:14.230035 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1173/2362] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 26 11:07:14 2022] PUT /identity/v3/projects/435c67c00aae4e47892622bfde5f2db3/users/0ae4787c81e944ad9c9197ae8eec26b0/roles/1f97e4602e2e4d6498913a81f9ae8d6a => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 26 11:07:14.232435 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bceec9f5-a7b2-46fa-af68-327662a2a040 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:07:14.247850 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bceec9f5-a7b2-46fa-af68-327662a2a040 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:07:14.326643 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ddc73a6-98d1-42b3-9291-c98479724797 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:07:14.327328 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddc73a6-98d1-42b3-9291-c98479724797 admin admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:07:14.327572 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddc73a6-98d1-42b3-9291-c98479724797 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:07:14.327822 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddc73a6-98d1-42b3-9291-c98479724797 admin admin] PATH_INFO: `/v3/roles` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:07:14.328464 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ddc73a6-98d1-42b3-9291-c98479724797 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:07:14.330139 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ddc73a6-98d1-42b3-9291-c98479724797 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:07:14.344943 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1190/2363] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 26 11:07:14 2022] GET /identity/v3/roles => generated 1420 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 11:07:14.346576 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eadaa2a5-c7e7-46cf-9785-928ddd49e6ea admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:07:14.359249 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eadaa2a5-c7e7-46cf-9785-928ddd49e6ea admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:07:14.365147 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299768fe-9716-48f5-bd8a-54ef3228ffe6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:07:14.366058 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299768fe-9716-48f5-bd8a-54ef3228ffe6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:07:14.366526 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299768fe-9716-48f5-bd8a-54ef3228ffe6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:07:14.367030 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299768fe-9716-48f5-bd8a-54ef3228ffe6 admin admin] PATH_INFO: `/v3/projects/435c67c00aae4e47892622bfde5f2db3/users/0ae4787c81e944ad9c9197ae8eec26b0/roles/1f97e4602e2e4d6498913a81f9ae8d6a` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:07:14.369537 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-299768fe-9716-48f5-bd8a-54ef3228ffe6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=435c67c00aae4e47892622bfde5f2db3, user_id=0ae4787c81e944ad9c9197ae8eec26b0, role_id=1f97e4602e2e4d6498913a81f9ae8d6a)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:07:14.372505 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-299768fe-9716-48f5-bd8a-54ef3228ffe6 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:07:14.387998 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1174/2364] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 26 11:07:14 2022] PUT /identity/v3/projects/435c67c00aae4e47892622bfde5f2db3/users/0ae4787c81e944ad9c9197ae8eec26b0/roles/1f97e4602e2e4d6498913a81f9ae8d6a => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 11:07:14.399177 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93428dc-4f02-42ef-a59a-0e8fbb7919b2 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:07:14.399514 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93428dc-4f02-42ef-a59a-0e8fbb7919b2 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:07:14.399752 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93428dc-4f02-42ef-a59a-0e8fbb7919b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:07:14.450491 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-b93428dc-4f02-42ef-a59a-0e8fbb7919b2 None None] MFA Rules not processed for user `0ae4787c81e944ad9c9197ae8eec26b0`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 11:07:14.452584 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-b93428dc-4f02-42ef-a59a-0e8fbb7919b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:07:14.481202 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 11:07:14.598062 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1191/2365] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 11:07:14 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 11:07:14.612429 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299768fe-9716-48f5-bd8a-54ef3228ffe6 admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:07:14.615500 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-299768fe-9716-48f5-bd8a-54ef3228ffe6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:07:14.759587 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299768fe-9716-48f5-bd8a-54ef3228ffe6 admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:07:14.764208 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4adde5ee-08d1-4e84-9776-6754a4b3c0c8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c277338a7f814cc6b01c5419775031bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e30ccf0c1204727b786876c141dbcd2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:07:14.764671 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4adde5ee-08d1-4e84-9776-6754a4b3c0c8 service cinder] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:07:14.764955 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4adde5ee-08d1-4e84-9776-6754a4b3c0c8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:07:14.765215 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4adde5ee-08d1-4e84-9776-6754a4b3c0c8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:07:14.785511 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 11:07:14.882027 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4adde5ee-08d1-4e84-9776-6754a4b3c0c8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:07:14.883664 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4adde5ee-08d1-4e84-9776-6754a4b3c0c8 service cinder] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:07:14.914872 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1175/2366] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 11:07:14 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 11:07:17.833111 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93428dc-4f02-42ef-a59a-0e8fbb7919b2 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:07:17.994624 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93428dc-4f02-42ef-a59a-0e8fbb7919b2 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:07:18.005304 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29a4783-9a21-4843-9ab6-63e24d870c2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:07:18.006002 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29a4783-9a21-4843-9ab6-63e24d870c2e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:07:18.006582 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29a4783-9a21-4843-9ab6-63e24d870c2e admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:07:18.006799 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29a4783-9a21-4843-9ab6-63e24d870c2e admin admin] PATH_INFO: `/v3/users/37c7cab523864c9796690f4b20ce95ac` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:07:18.008212 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a29a4783-9a21-4843-9ab6-63e24d870c2e admin admin] RBAC: Authorizing `identity:delete_user(user_id=37c7cab523864c9796690f4b20ce95ac)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:07:18.013131 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a29a4783-9a21-4843-9ab6-63e24d870c2e admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:07:18.109544 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-a29a4783-9a21-4843-9ab6-63e24d870c2e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '37c7cab523864c9796690f4b20ce95ac'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:07:18.110327 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-a29a4783-9a21-4843-9ab6-63e24d870c2e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '37c7cab523864c9796690f4b20ce95ac'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:07:18.129449 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-a29a4783-9a21-4843-9ab6-63e24d870c2e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '37c7cab523864c9796690f4b20ce95ac'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:07:18.131451 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-a29a4783-9a21-4843-9ab6-63e24d870c2e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '37c7cab523864c9796690f4b20ce95ac'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:07:18.181369 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-a29a4783-9a21-4843-9ab6-63e24d870c2e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '37c7cab523864c9796690f4b20ce95ac'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:07:18.216160 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1192/2367] 10.222.0.40 () {58 vars in 1224 bytes} [Fri Aug 26 11:07:17 2022] DELETE /identity/v3/users/37c7cab523864c9796690f4b20ce95ac => generated 0 bytes in 388 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 11:07:18.222181 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd24f5bb-74b5-4923-9dc5-b4648fa73ac2 None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:07:18.222181 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd24f5bb-74b5-4923-9dc5-b4648fa73ac2 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:07:18.222181 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd24f5bb-74b5-4923-9dc5-b4648fa73ac2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:07:18.266528 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-cd24f5bb-74b5-4923-9dc5-b4648fa73ac2 None None] MFA Rules not processed for user `ca155a48e2024c32af3fd2e19e5fac11`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 11:07:18.268460 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-cd24f5bb-74b5-4923-9dc5-b4648fa73ac2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:07:18.406823 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1176/2368] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 11:07:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 11:07:18.418516 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29a4783-9a21-4843-9ab6-63e24d870c2e admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:07:18.420786 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-a29a4783-9a21-4843-9ab6-63e24d870c2e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:07:18.515732 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 11:07:18.563148 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29a4783-9a21-4843-9ab6-63e24d870c2e admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:07:18.574836 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a7b640-7706-4958-aa00-8b399319b9f7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91da2acd2534423091537e1ba068d2b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e30ccf0c1204727b786876c141dbcd2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:07:18.575485 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a7b640-7706-4958-aa00-8b399319b9f7 service neutron] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:07:18.575768 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a7b640-7706-4958-aa00-8b399319b9f7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:07:18.576042 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a7b640-7706-4958-aa00-8b399319b9f7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:07:18.596528 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a7b640-7706-4958-aa00-8b399319b9f7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:07:18.598498 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a7b640-7706-4958-aa00-8b399319b9f7 service neutron] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:07:18.611941 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1193/2369] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 11:07:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 194 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 11:07:19.068681 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd24f5bb-74b5-4923-9dc5-b4648fa73ac2 None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:07:19.070404 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-cd24f5bb-74b5-4923-9dc5-b4648fa73ac2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:07:19.084102 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd24f5bb-74b5-4923-9dc5-b4648fa73ac2 None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:07:19.099184 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74fe728-e151-40be-acff-d9ea6dd388c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:07:19.100298 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74fe728-e151-40be-acff-d9ea6dd388c1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:07:19.100584 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74fe728-e151-40be-acff-d9ea6dd388c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:07:19.100935 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74fe728-e151-40be-acff-d9ea6dd388c1 admin admin] PATH_INFO: `/v3/projects/7182450761da4e99a374be8630789d72` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:07:19.102034 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c74fe728-e151-40be-acff-d9ea6dd388c1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7182450761da4e99a374be8630789d72)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:07:19.104449 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c74fe728-e151-40be-acff-d9ea6dd388c1 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:07:19.168945 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-c74fe728-e151-40be-acff-d9ea6dd388c1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7182450761da4e99a374be8630789d72', 'request_id': 'req-c74fe728-e151-40be-acff-d9ea6dd388c1'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:07:19.177448 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-c74fe728-e151-40be-acff-d9ea6dd388c1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7182450761da4e99a374be8630789d72', 'request_id': 'req-c74fe728-e151-40be-acff-d9ea6dd388c1'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:07:19.187578 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-c74fe728-e151-40be-acff-d9ea6dd388c1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7182450761da4e99a374be8630789d72', 'request_id': 'req-c74fe728-e151-40be-acff-d9ea6dd388c1'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:07:19.208419 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-c74fe728-e151-40be-acff-d9ea6dd388c1 admin admin] The token cache is being invalidate because project 7182450761da4e99a374be8630789d72 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81423) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 11:07:19.208705 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-c74fe728-e151-40be-acff-d9ea6dd388c1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:07:19.209371 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-c74fe728-e151-40be-acff-d9ea6dd388c1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:07:19.213717 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1177/2370] 10.222.0.40 () {58 vars in 1233 bytes} [Fri Aug 26 11:07:19 2022] DELETE /identity/v3/projects/7182450761da4e99a374be8630789d72 => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 11:07:19.214946 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a7b640-7706-4958-aa00-8b399319b9f7 service neutron] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:07:19.225590 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a7b640-7706-4958-aa00-8b399319b9f7 service neutron] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:07:19.304225 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ab91b4-2442-40cc-9489-9e4a07c7fead admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:07:19.305010 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ab91b4-2442-40cc-9489-9e4a07c7fead admin admin] REQUEST_METHOD: `DELETE` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:07:19.305501 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ab91b4-2442-40cc-9489-9e4a07c7fead admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:07:19.305826 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ab91b4-2442-40cc-9489-9e4a07c7fead admin admin] PATH_INFO: `/v3/users/0ae4787c81e944ad9c9197ae8eec26b0` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:07:19.307073 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ab91b4-2442-40cc-9489-9e4a07c7fead admin admin] RBAC: Authorizing `identity:delete_user(user_id=0ae4787c81e944ad9c9197ae8eec26b0)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:07:19.309019 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ab91b4-2442-40cc-9489-9e4a07c7fead admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:07:19.379668 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-12ab91b4-2442-40cc-9489-9e4a07c7fead admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0ae4787c81e944ad9c9197ae8eec26b0'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:07:19.381306 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-12ab91b4-2442-40cc-9489-9e4a07c7fead admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0ae4787c81e944ad9c9197ae8eec26b0'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:07:19.399191 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-12ab91b4-2442-40cc-9489-9e4a07c7fead admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0ae4787c81e944ad9c9197ae8eec26b0'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:07:19.401149 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-12ab91b4-2442-40cc-9489-9e4a07c7fead admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0ae4787c81e944ad9c9197ae8eec26b0'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:07:19.430424 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-12ab91b4-2442-40cc-9489-9e4a07c7fead admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0ae4787c81e944ad9c9197ae8eec26b0'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:07:19.453307 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1194/2371] 10.222.0.40 () {58 vars in 1224 bytes} [Fri Aug 26 11:07:19 2022] DELETE /identity/v3/users/0ae4787c81e944ad9c9197ae8eec26b0 => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 11:07:20.034658 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74fe728-e151-40be-acff-d9ea6dd388c1 admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:07:20.037586 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-c74fe728-e151-40be-acff-d9ea6dd388c1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:07:20.062452 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74fe728-e151-40be-acff-d9ea6dd388c1 admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:07:20.137812 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 11:07:20.179439 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd7a091-1fac-4508-a705-d2f48556e699 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:07:20.186348 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd7a091-1fac-4508-a705-d2f48556e699 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:07:20.186720 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd7a091-1fac-4508-a705-d2f48556e699 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:07:20.187171 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd7a091-1fac-4508-a705-d2f48556e699 admin admin] PATH_INFO: `/v3/projects/435c67c00aae4e47892622bfde5f2db3` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:07:20.189528 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd7a091-1fac-4508-a705-d2f48556e699 admin admin] RBAC: Authorizing `identity:delete_project(project_id=435c67c00aae4e47892622bfde5f2db3)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:07:20.192540 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd7a091-1fac-4508-a705-d2f48556e699 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:07:20.273630 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-fdd7a091-1fac-4508-a705-d2f48556e699 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '435c67c00aae4e47892622bfde5f2db3', 'request_id': 'req-fdd7a091-1fac-4508-a705-d2f48556e699'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:07:20.302241 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-fdd7a091-1fac-4508-a705-d2f48556e699 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '435c67c00aae4e47892622bfde5f2db3', 'request_id': 'req-fdd7a091-1fac-4508-a705-d2f48556e699'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:07:20.309886 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-fdd7a091-1fac-4508-a705-d2f48556e699 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '435c67c00aae4e47892622bfde5f2db3', 'request_id': 'req-fdd7a091-1fac-4508-a705-d2f48556e699'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:07:20.330940 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-fdd7a091-1fac-4508-a705-d2f48556e699 admin admin] The token cache is being invalidate because project 435c67c00aae4e47892622bfde5f2db3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81423) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 11:07:20.331374 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-fdd7a091-1fac-4508-a705-d2f48556e699 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:07:20.332205 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-fdd7a091-1fac-4508-a705-d2f48556e699 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:07:20.338013 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1178/2372] 10.222.0.40 () {58 vars in 1233 bytes} [Fri Aug 26 11:07:20 2022] DELETE /identity/v3/projects/435c67c00aae4e47892622bfde5f2db3 => generated 0 bytes in 301 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 11:07:22.313208 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6977854d-e718-4ffb-9431-114b265d7cfd None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:07:22.313704 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6977854d-e718-4ffb-9431-114b265d7cfd None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:07:22.313815 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6977854d-e718-4ffb-9431-114b265d7cfd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:07:22.336275 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-6977854d-e718-4ffb-9431-114b265d7cfd None None] MFA Rules not processed for user `50be12ccbd274a8792615e9dd10b61ca`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 11:07:22.337485 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-6977854d-e718-4ffb-9431-114b265d7cfd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:07:22.518738 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 11:07:22.530129 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1195/2373] 10.222.0.40 () {58 vars in 1009 bytes} [Fri Aug 26 11:07:22 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 11:07:22.541082 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd7a091-1fac-4508-a705-d2f48556e699 admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:07:22.542548 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-fdd7a091-1fac-4508-a705-d2f48556e699 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:07:22.615253 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd7a091-1fac-4508-a705-d2f48556e699 admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:07:22.620747 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730bcc0e-2b92-4b72-a120-9a95660b0683 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6529bea296c9482a973ced5fe9f2212f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e30ccf0c1204727b786876c141dbcd2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:07:22.621484 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730bcc0e-2b92-4b72-a120-9a95660b0683 service swift] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:07:22.621904 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730bcc0e-2b92-4b72-a120-9a95660b0683 service swift] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:07:22.622136 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730bcc0e-2b92-4b72-a120-9a95660b0683 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:07:22.630621 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-730bcc0e-2b92-4b72-a120-9a95660b0683 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:07:22.632211 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-730bcc0e-2b92-4b72-a120-9a95660b0683 service swift] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:07:22.641134 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1179/2374] 10.222.0.40 () {60 vars in 1384 bytes} [Fri Aug 26 11:07:22 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 26 11:07:23.192788 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3d032d-7ca2-4ec2-ad42-472f147389ca None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:07:23.193064 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3d032d-7ca2-4ec2-ad42-472f147389ca None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:07:23.193334 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3d032d-7ca2-4ec2-ad42-472f147389ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:07:23.212263 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-5f3d032d-7ca2-4ec2-ad42-472f147389ca None None] MFA Rules not processed for user `50be12ccbd274a8792615e9dd10b61ca`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 11:07:23.213486 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-5f3d032d-7ca2-4ec2-ad42-472f147389ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:07:23.220874 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1196/2375] 10.222.0.40 () {58 vars in 967 bytes} [Fri Aug 26 11:07:23 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 11:07:23.233535 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730bcc0e-2b92-4b72-a120-9a95660b0683 service swift] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:07:23.245281 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730bcc0e-2b92-4b72-a120-9a95660b0683 service swift] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:07:23.250294 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2fd19a-1aa3-477d-9cf5-4d56e9e7e35e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6529bea296c9482a973ced5fe9f2212f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e30ccf0c1204727b786876c141dbcd2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:07:23.250896 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2fd19a-1aa3-477d-9cf5-4d56e9e7e35e service swift] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:07:23.251234 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2fd19a-1aa3-477d-9cf5-4d56e9e7e35e service swift] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:07:23.251446 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2fd19a-1aa3-477d-9cf5-4d56e9e7e35e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:07:23.261056 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e2fd19a-1aa3-477d-9cf5-4d56e9e7e35e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:07:23.262698 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e2fd19a-1aa3-477d-9cf5-4d56e9e7e35e service swift] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:07:23.271016 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1180/2376] 10.222.0.40 () {60 vars in 1384 bytes} [Fri Aug 26 11:07:23 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 26 11:07:23.526282 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc98fa9-8dd8-4317-958d-58e8e0b59052 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:07:23.527098 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc98fa9-8dd8-4317-958d-58e8e0b59052 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:07:23.527424 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc98fa9-8dd8-4317-958d-58e8e0b59052 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:07:23.547544 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-ecc98fa9-8dd8-4317-958d-58e8e0b59052 None None] MFA Rules not processed for user `50be12ccbd274a8792615e9dd10b61ca`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 11:07:23.548756 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-ecc98fa9-8dd8-4317-958d-58e8e0b59052 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:07:23.557197 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1197/2377] 10.222.0.40 () {58 vars in 1009 bytes} [Fri Aug 26 11:07:23 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 11:07:23.570457 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2fd19a-1aa3-477d-9cf5-4d56e9e7e35e service swift] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:07:23.586858 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2fd19a-1aa3-477d-9cf5-4d56e9e7e35e service swift] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:07:23.592158 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15319dc-9c5e-4e47-b299-c440cb349bf7 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6529bea296c9482a973ced5fe9f2212f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e30ccf0c1204727b786876c141dbcd2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:07:23.592342 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15319dc-9c5e-4e47-b299-c440cb349bf7 service swift] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:07:23.592551 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15319dc-9c5e-4e47-b299-c440cb349bf7 service swift] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:07:23.592829 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15319dc-9c5e-4e47-b299-c440cb349bf7 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:07:23.603912 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15319dc-9c5e-4e47-b299-c440cb349bf7 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:07:23.605524 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15319dc-9c5e-4e47-b299-c440cb349bf7 service swift] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:07:23.615585 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1181/2378] 10.222.0.40 () {60 vars in 1384 bytes} [Fri Aug 26 11:07:23 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 26 11:07:24.013589 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b287713-dbbd-460b-973f-dffb2b5b8514 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:07:24.013870 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b287713-dbbd-460b-973f-dffb2b5b8514 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:07:24.014169 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b287713-dbbd-460b-973f-dffb2b5b8514 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:07:24.033458 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-7b287713-dbbd-460b-973f-dffb2b5b8514 None None] MFA Rules not processed for user `50be12ccbd274a8792615e9dd10b61ca`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 11:07:24.034602 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-7b287713-dbbd-460b-973f-dffb2b5b8514 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:07:24.041764 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1198/2379] 10.222.0.40 () {58 vars in 967 bytes} [Fri Aug 26 11:07:24 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 11:07:24.055046 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15319dc-9c5e-4e47-b299-c440cb349bf7 service swift] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:07:24.065619 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15319dc-9c5e-4e47-b299-c440cb349bf7 service swift] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:07:24.071594 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6fca42-7105-4a69-a004-3a2a408bb245 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6529bea296c9482a973ced5fe9f2212f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e30ccf0c1204727b786876c141dbcd2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:07:24.072210 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6fca42-7105-4a69-a004-3a2a408bb245 service swift] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:07:24.072508 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6fca42-7105-4a69-a004-3a2a408bb245 service swift] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:07:24.072781 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6fca42-7105-4a69-a004-3a2a408bb245 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:07:24.080804 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b6fca42-7105-4a69-a004-3a2a408bb245 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:07:24.083031 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b6fca42-7105-4a69-a004-3a2a408bb245 service swift] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:07:24.091454 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1182/2380] 10.222.0.40 () {60 vars in 1384 bytes} [Fri Aug 26 11:07:24 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 26 11:07:26.226390 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b287713-dbbd-460b-973f-dffb2b5b8514 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:07:26.227860 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-7b287713-dbbd-460b-973f-dffb2b5b8514 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:07:26.303434 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b287713-dbbd-460b-973f-dffb2b5b8514 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:07:26.313898 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b55ea8e-9fae-4e22-a1dd-66485ef2704c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:07:26.314739 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b55ea8e-9fae-4e22-a1dd-66485ef2704c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:07:26.315007 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b55ea8e-9fae-4e22-a1dd-66485ef2704c admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:07:26.315274 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b55ea8e-9fae-4e22-a1dd-66485ef2704c admin admin] PATH_INFO: `/v3/users/275df87a6e384102be2194b74b180358` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:07:26.316434 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b55ea8e-9fae-4e22-a1dd-66485ef2704c admin admin] RBAC: Authorizing `identity:delete_user(user_id=275df87a6e384102be2194b74b180358)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:07:26.319021 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b55ea8e-9fae-4e22-a1dd-66485ef2704c admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:07:26.371309 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-2b55ea8e-9fae-4e22-a1dd-66485ef2704c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '275df87a6e384102be2194b74b180358'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:07:26.371882 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-2b55ea8e-9fae-4e22-a1dd-66485ef2704c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '275df87a6e384102be2194b74b180358'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:07:26.378827 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-2b55ea8e-9fae-4e22-a1dd-66485ef2704c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '275df87a6e384102be2194b74b180358'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:07:26.380270 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-2b55ea8e-9fae-4e22-a1dd-66485ef2704c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '275df87a6e384102be2194b74b180358'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:07:26.392432 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-2b55ea8e-9fae-4e22-a1dd-66485ef2704c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '275df87a6e384102be2194b74b180358'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:07:26.410142 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1199/2381] 10.222.0.40 () {58 vars in 1224 bytes} [Fri Aug 26 11:07:26 2022] DELETE /identity/v3/users/275df87a6e384102be2194b74b180358 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 11:07:26.416063 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a30af20-c7cc-4b9d-a44f-ea9ef4ded345 None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:07:26.416587 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a30af20-c7cc-4b9d-a44f-ea9ef4ded345 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:07:26.416682 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a30af20-c7cc-4b9d-a44f-ea9ef4ded345 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:07:26.443607 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-3a30af20-c7cc-4b9d-a44f-ea9ef4ded345 None None] MFA Rules not processed for user `ca155a48e2024c32af3fd2e19e5fac11`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 11:07:26.445763 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-3a30af20-c7cc-4b9d-a44f-ea9ef4ded345 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:07:26.459566 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 11:07:26.530951 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1183/2382] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 11:07:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 11:07:26.542242 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b55ea8e-9fae-4e22-a1dd-66485ef2704c admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:07:26.543656 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-2b55ea8e-9fae-4e22-a1dd-66485ef2704c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:07:26.614092 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b55ea8e-9fae-4e22-a1dd-66485ef2704c admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:07:26.617868 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22747438-3965-4efb-97f6-f17ce2359cdb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91da2acd2534423091537e1ba068d2b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e30ccf0c1204727b786876c141dbcd2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:07:26.618791 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22747438-3965-4efb-97f6-f17ce2359cdb service neutron] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:07:26.619067 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22747438-3965-4efb-97f6-f17ce2359cdb service neutron] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:07:26.619336 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22747438-3965-4efb-97f6-f17ce2359cdb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:07:26.630014 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22747438-3965-4efb-97f6-f17ce2359cdb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:07:26.631653 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22747438-3965-4efb-97f6-f17ce2359cdb service neutron] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:07:26.643537 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1200/2383] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 11:07:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 11:07:26.872308 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a30af20-c7cc-4b9d-a44f-ea9ef4ded345 None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:07:26.876803 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-3a30af20-c7cc-4b9d-a44f-ea9ef4ded345 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:07:26.898046 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a30af20-c7cc-4b9d-a44f-ea9ef4ded345 None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:07:26.904472 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9013e646-003b-4490-b14c-053e6671ce7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:07:26.905143 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9013e646-003b-4490-b14c-053e6671ce7b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:07:26.905421 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9013e646-003b-4490-b14c-053e6671ce7b admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:07:26.905697 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9013e646-003b-4490-b14c-053e6671ce7b admin admin] PATH_INFO: `/v3/projects/1b87c51628c342cfa953a7110735d469` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:07:26.907069 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9013e646-003b-4490-b14c-053e6671ce7b admin admin] RBAC: Authorizing `identity:delete_project(project_id=1b87c51628c342cfa953a7110735d469)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:07:26.909179 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9013e646-003b-4490-b14c-053e6671ce7b admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:07:26.974128 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-9013e646-003b-4490-b14c-053e6671ce7b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1b87c51628c342cfa953a7110735d469', 'request_id': 'req-9013e646-003b-4490-b14c-053e6671ce7b'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:07:26.980661 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-9013e646-003b-4490-b14c-053e6671ce7b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1b87c51628c342cfa953a7110735d469', 'request_id': 'req-9013e646-003b-4490-b14c-053e6671ce7b'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:07:26.990650 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-9013e646-003b-4490-b14c-053e6671ce7b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1b87c51628c342cfa953a7110735d469', 'request_id': 'req-9013e646-003b-4490-b14c-053e6671ce7b'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:07:27.008211 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-9013e646-003b-4490-b14c-053e6671ce7b admin admin] The token cache is being invalidate because project 1b87c51628c342cfa953a7110735d469 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81423) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 11:07:27.008527 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-9013e646-003b-4490-b14c-053e6671ce7b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:07:27.009223 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-9013e646-003b-4490-b14c-053e6671ce7b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:07:27.011563 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1184/2384] 10.222.0.40 () {58 vars in 1233 bytes} [Fri Aug 26 11:07:26 2022] DELETE /identity/v3/projects/1b87c51628c342cfa953a7110735d469 => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 11:07:27.016724 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36618736-2ebb-4aad-b247-b018848b2f4b None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:07:27.017034 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36618736-2ebb-4aad-b247-b018848b2f4b None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:07:27.017338 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36618736-2ebb-4aad-b247-b018848b2f4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:07:27.028015 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 11:07:27.052225 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-36618736-2ebb-4aad-b247-b018848b2f4b None None] MFA Rules not processed for user `ca155a48e2024c32af3fd2e19e5fac11`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 11:07:27.053811 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-36618736-2ebb-4aad-b247-b018848b2f4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:07:27.148840 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1201/2385] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 11:07:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 11:07:27.158648 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0446df59-8225-471f-85f7-e606c660aa7c None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:07:27.158947 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0446df59-8225-471f-85f7-e606c660aa7c None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:07:27.159101 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0446df59-8225-471f-85f7-e606c660aa7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:07:27.193233 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-0446df59-8225-471f-85f7-e606c660aa7c None None] MFA Rules not processed for user `ca155a48e2024c32af3fd2e19e5fac11`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 11:07:27.195337 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-0446df59-8225-471f-85f7-e606c660aa7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:07:27.205834 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1185/2386] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 11:07:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 11:07:27.211115 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36618736-2ebb-4aad-b247-b018848b2f4b None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:07:27.227436 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36618736-2ebb-4aad-b247-b018848b2f4b None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:07:27.233099 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e7be51-6af0-42a8-afad-e5f1761ef275 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:07:27.234193 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e7be51-6af0-42a8-afad-e5f1761ef275 admin admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:07:27.234515 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e7be51-6af0-42a8-afad-e5f1761ef275 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:07:27.234999 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e7be51-6af0-42a8-afad-e5f1761ef275 admin admin] PATH_INFO: `/v3/domains` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:07:27.236057 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e7be51-6af0-42a8-afad-e5f1761ef275 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 11:07:27.236359 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e7be51-6af0-42a8-afad-e5f1761ef275 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:07:27.238094 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e7be51-6af0-42a8-afad-e5f1761ef275 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:07:27.268245 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1202/2387] 10.222.0.40 () {58 vars in 1153 bytes} [Fri Aug 26 11:07:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 11:07:27.274438 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0446df59-8225-471f-85f7-e606c660aa7c None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:07:27.288090 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0446df59-8225-471f-85f7-e606c660aa7c None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:07:27.293593 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03890801-9992-49d6-98e2-0dff1de180e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:07:27.294389 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03890801-9992-49d6-98e2-0dff1de180e0 admin admin] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:07:27.294835 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03890801-9992-49d6-98e2-0dff1de180e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:07:27.295060 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03890801-9992-49d6-98e2-0dff1de180e0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:07:27.295768 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03890801-9992-49d6-98e2-0dff1de180e0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:07:27.299439 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03890801-9992-49d6-98e2-0dff1de180e0 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:07:27.345913 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1186/2388] 10.222.0.40 () {60 vars in 1153 bytes} [Fri Aug 26 11:07:27 2022] POST /identity/v3/projects => generated 381 bytes in 73 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 11:07:27.352141 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e7be51-6af0-42a8-afad-e5f1761ef275 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:07:27.368765 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e7be51-6af0-42a8-afad-e5f1761ef275 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:07:27.448270 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf8daaf-62e2-44ed-bc53-86527a9bc697 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:07:27.448889 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf8daaf-62e2-44ed-bc53-86527a9bc697 admin admin] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:07:27.449162 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf8daaf-62e2-44ed-bc53-86527a9bc697 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:07:27.449430 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf8daaf-62e2-44ed-bc53-86527a9bc697 admin admin] PATH_INFO: `/v3/users` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:07:27.450094 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf8daaf-62e2-44ed-bc53-86527a9bc697 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:07:27.452084 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf8daaf-62e2-44ed-bc53-86527a9bc697 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:07:27.482186 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1203/2389] 10.222.0.40 () {60 vars in 1144 bytes} [Fri Aug 26 11:07:27 2022] POST /identity/v3/users => generated 291 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 26 11:07:27.486605 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03890801-9992-49d6-98e2-0dff1de180e0 admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:07:27.503251 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03890801-9992-49d6-98e2-0dff1de180e0 admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:07:27.511251 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54667bbb-016c-468a-a073-e3f4a96bc8e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:07:27.512013 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54667bbb-016c-468a-a073-e3f4a96bc8e5 admin admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:07:27.512362 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54667bbb-016c-468a-a073-e3f4a96bc8e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:07:27.512709 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54667bbb-016c-468a-a073-e3f4a96bc8e5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:07:27.513520 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54667bbb-016c-468a-a073-e3f4a96bc8e5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:07:27.515457 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54667bbb-016c-468a-a073-e3f4a96bc8e5 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:07:27.529781 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1187/2390] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 26 11:07:27 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 11:07:27.534179 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf8daaf-62e2-44ed-bc53-86527a9bc697 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:07:27.545963 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf8daaf-62e2-44ed-bc53-86527a9bc697 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:07:27.552324 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14302b54-a395-4ab3-8446-2ff2331a69cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:07:27.552924 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14302b54-a395-4ab3-8446-2ff2331a69cb admin admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:07:27.553223 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14302b54-a395-4ab3-8446-2ff2331a69cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:07:27.553483 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14302b54-a395-4ab3-8446-2ff2331a69cb admin admin] PATH_INFO: `/v3/roles` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:07:27.554422 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14302b54-a395-4ab3-8446-2ff2331a69cb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:07:27.556746 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14302b54-a395-4ab3-8446-2ff2331a69cb admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:07:27.577880 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1204/2391] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 26 11:07:27 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 11:07:27.579121 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54667bbb-016c-468a-a073-e3f4a96bc8e5 admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:07:27.590890 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54667bbb-016c-468a-a073-e3f4a96bc8e5 admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:07:27.595681 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f054e4-8fa6-457e-8f6b-f2482eac4f29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:07:27.596347 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f054e4-8fa6-457e-8f6b-f2482eac4f29 admin admin] REQUEST_METHOD: `PUT` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:07:27.596625 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f054e4-8fa6-457e-8f6b-f2482eac4f29 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:07:27.596902 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f054e4-8fa6-457e-8f6b-f2482eac4f29 admin admin] PATH_INFO: `/v3/projects/de754769f766487e859de1729e9d7a77/users/c3aaf776da3a4bb0853946c195112f40/roles/23e174ce743b46eabe618c212691da55` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:07:27.621637 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f054e4-8fa6-457e-8f6b-f2482eac4f29 admin admin] RBAC: Authorizing `identity:create_grant(project_id=de754769f766487e859de1729e9d7a77, user_id=c3aaf776da3a4bb0853946c195112f40, role_id=23e174ce743b46eabe618c212691da55)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:07:27.626410 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f054e4-8fa6-457e-8f6b-f2482eac4f29 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:07:27.651660 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1188/2392] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 26 11:07:27 2022] PUT /identity/v3/projects/de754769f766487e859de1729e9d7a77/users/c3aaf776da3a4bb0853946c195112f40/roles/23e174ce743b46eabe618c212691da55 => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 11:07:27.655926 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad28bb70-012d-41a4-a2c9-2ee481ce1f2b None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:07:27.656207 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad28bb70-012d-41a4-a2c9-2ee481ce1f2b None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:07:27.656478 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad28bb70-012d-41a4-a2c9-2ee481ce1f2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:07:27.684363 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-ad28bb70-012d-41a4-a2c9-2ee481ce1f2b None None] MFA Rules not processed for user `ca155a48e2024c32af3fd2e19e5fac11`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 11:07:27.685268 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-ad28bb70-012d-41a4-a2c9-2ee481ce1f2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:07:27.802806 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1205/2393] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 11:07:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 11:07:27.814123 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f054e4-8fa6-457e-8f6b-f2482eac4f29 admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:07:27.816838 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-f7f054e4-8fa6-457e-8f6b-f2482eac4f29 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:07:28.006464 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 11:07:28.026945 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f054e4-8fa6-457e-8f6b-f2482eac4f29 admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:07:28.030556 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed732fe-2b50-415d-8515-d2a00142c3fa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91da2acd2534423091537e1ba068d2b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e30ccf0c1204727b786876c141dbcd2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:07:28.039061 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed732fe-2b50-415d-8515-d2a00142c3fa service neutron] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:07:28.039061 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed732fe-2b50-415d-8515-d2a00142c3fa service neutron] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:07:28.039061 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed732fe-2b50-415d-8515-d2a00142c3fa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:07:28.044316 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ed732fe-2b50-415d-8515-d2a00142c3fa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:07:28.045830 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ed732fe-2b50-415d-8515-d2a00142c3fa service neutron] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:07:28.058882 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1189/2394] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 11:07:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 246 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 26 11:07:28.360125 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b66a289-0a98-497c-82ff-19a18633652b None None] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:07:28.360954 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b66a289-0a98-497c-82ff-19a18633652b None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:07:28.361086 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b66a289-0a98-497c-82ff-19a18633652b None None] PATH_INFO: `/` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:07:28.364652 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1206/2395] 10.222.0.40 () {54 vars in 921 bytes} [Fri Aug 26 11:07:28 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 11:07:28.375859 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075b8a1e-a288-46c1-a755-090e39c6e014 None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:07:28.375859 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075b8a1e-a288-46c1-a755-090e39c6e014 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:07:28.376058 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075b8a1e-a288-46c1-a755-090e39c6e014 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:07:28.407249 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-075b8a1e-a288-46c1-a755-090e39c6e014 None None] MFA Rules not processed for user `1f4b4a662db049ecacc865ecff10bdb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 11:07:28.411501 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-075b8a1e-a288-46c1-a755-090e39c6e014 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:07:28.499247 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1190/2396] 10.222.0.40 () {58 vars in 1018 bytes} [Fri Aug 26 11:07:28 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 11:07:28.511989 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b66a289-0a98-497c-82ff-19a18633652b None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:07:28.514869 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-8b66a289-0a98-497c-82ff-19a18633652b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:07:28.531356 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 11:07:28.595247 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b66a289-0a98-497c-82ff-19a18633652b None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:07:28.600933 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c0148c-f604-4204-a1f2-011ef80bd384 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8631d7ecfd2849daa8694aecccd22a05', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e30ccf0c1204727b786876c141dbcd2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:07:28.601788 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c0148c-f604-4204-a1f2-011ef80bd384 service glance] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:07:28.602164 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c0148c-f604-4204-a1f2-011ef80bd384 service glance] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:07:28.602541 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c0148c-f604-4204-a1f2-011ef80bd384 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:07:28.621245 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c0148c-f604-4204-a1f2-011ef80bd384 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:07:28.622731 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c0148c-f604-4204-a1f2-011ef80bd384 service glance] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:07:28.639141 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1207/2397] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 11:07:28 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 11:07:28.815325 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9813a23-0acc-47b1-8e7e-cebcc69d5800 None None] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:07:28.815657 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9813a23-0acc-47b1-8e7e-cebcc69d5800 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:07:28.815859 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9813a23-0acc-47b1-8e7e-cebcc69d5800 None None] PATH_INFO: `/` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:07:28.818318 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1191/2398] 10.222.0.40 () {54 vars in 921 bytes} [Fri Aug 26 11:07:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 11:07:28.825512 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059ab2fe-e5fb-4e3d-bba3-26a50f84a5ad None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:07:28.825769 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9813a23-0acc-47b1-8e7e-cebcc69d5800 None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:07:28.825940 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059ab2fe-e5fb-4e3d-bba3-26a50f84a5ad None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:07:28.826332 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059ab2fe-e5fb-4e3d-bba3-26a50f84a5ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:07:28.830516 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-b9813a23-0acc-47b1-8e7e-cebcc69d5800 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:07:28.851889 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9813a23-0acc-47b1-8e7e-cebcc69d5800 None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:07:28.865172 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-059ab2fe-e5fb-4e3d-bba3-26a50f84a5ad None None] MFA Rules not processed for user `91da2acd2534423091537e1ba068d2b7`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 11:07:28.866588 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301437b1-6c43-46f1-9c9e-02f39d5862d7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91da2acd2534423091537e1ba068d2b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e30ccf0c1204727b786876c141dbcd2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:07:28.866871 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-059ab2fe-e5fb-4e3d-bba3-26a50f84a5ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:07:28.867374 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301437b1-6c43-46f1-9c9e-02f39d5862d7 service neutron] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:07:28.867778 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301437b1-6c43-46f1-9c9e-02f39d5862d7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:07:28.868273 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301437b1-6c43-46f1-9c9e-02f39d5862d7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:07:28.885280 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-301437b1-6c43-46f1-9c9e-02f39d5862d7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:07:28.889786 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1208/2399] 10.222.0.40 () {58 vars in 1018 bytes} [Fri Aug 26 11:07:28 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 66 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 11:07:28.908389 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059ab2fe-e5fb-4e3d-bba3-26a50f84a5ad None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:07:28.924000 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059ab2fe-e5fb-4e3d-bba3-26a50f84a5ad None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:07:28.929042 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fdee72f-cd2a-4308-a3c0-fedd6e41c936 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91da2acd2534423091537e1ba068d2b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e30ccf0c1204727b786876c141dbcd2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:07:28.930039 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fdee72f-cd2a-4308-a3c0-fedd6e41c936 service neutron] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:07:28.930561 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fdee72f-cd2a-4308-a3c0-fedd6e41c936 service neutron] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:07:28.930984 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fdee72f-cd2a-4308-a3c0-fedd6e41c936 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:07:28.941920 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fdee72f-cd2a-4308-a3c0-fedd6e41c936 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:07:28.946221 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fdee72f-cd2a-4308-a3c0-fedd6e41c936 service neutron] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:07:28.963312 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1209/2400] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 11:07:28 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 26 11:07:28.989552 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-301437b1-6c43-46f1-9c9e-02f39d5862d7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:07:28.991312 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-301437b1-6c43-46f1-9c9e-02f39d5862d7 service neutron] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:07:29.006046 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1192/2401] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 11:07:28 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 182 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 11:07:29.072669 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d05f5d-9045-482f-a5e6-b07112e00bdf None None] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:07:29.073157 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d05f5d-9045-482f-a5e6-b07112e00bdf None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:07:29.073527 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d05f5d-9045-482f-a5e6-b07112e00bdf None None] PATH_INFO: `/` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:07:29.075007 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1210/2402] 10.222.0.40 () {54 vars in 921 bytes} [Fri Aug 26 11:07:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 11:07:29.080983 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef996b9a-f362-449f-aedf-19915f209035 None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:07:29.081354 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef996b9a-f362-449f-aedf-19915f209035 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:07:29.081701 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef996b9a-f362-449f-aedf-19915f209035 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:07:29.106281 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-ef996b9a-f362-449f-aedf-19915f209035 None None] MFA Rules not processed for user `1f4b4a662db049ecacc865ecff10bdb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 11:07:29.107846 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-ef996b9a-f362-449f-aedf-19915f209035 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:07:29.125312 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1193/2403] 10.222.0.40 () {58 vars in 1018 bytes} [Fri Aug 26 11:07:29 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 11:07:29.140513 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d05f5d-9045-482f-a5e6-b07112e00bdf None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:07:29.161263 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d05f5d-9045-482f-a5e6-b07112e00bdf None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:07:29.166655 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487a374c-7313-4fef-898d-a3b27552a4e5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91da2acd2534423091537e1ba068d2b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e30ccf0c1204727b786876c141dbcd2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:07:29.167281 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487a374c-7313-4fef-898d-a3b27552a4e5 service neutron] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:07:29.167570 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487a374c-7313-4fef-898d-a3b27552a4e5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:07:29.167980 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487a374c-7313-4fef-898d-a3b27552a4e5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:07:29.199229 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-487a374c-7313-4fef-898d-a3b27552a4e5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:07:29.199229 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-487a374c-7313-4fef-898d-a3b27552a4e5 service neutron] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:07:29.201445 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84371161-0644-4ac1-9bd5-b920b83b158c None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:07:29.201721 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84371161-0644-4ac1-9bd5-b920b83b158c None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:07:29.202016 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84371161-0644-4ac1-9bd5-b920b83b158c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:07:29.221708 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1211/2404] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 11:07:29 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 26 11:07:29.240322 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-84371161-0644-4ac1-9bd5-b920b83b158c None None] MFA Rules not processed for user `c3aaf776da3a4bb0853946c195112f40`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 11:07:29.241732 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-84371161-0644-4ac1-9bd5-b920b83b158c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:07:29.310464 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 11:07:29.344979 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1194/2405] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 11:07:29 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 11:07:29.353475 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487a374c-7313-4fef-898d-a3b27552a4e5 service neutron] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:07:29.355218 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-487a374c-7313-4fef-898d-a3b27552a4e5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:07:29.441300 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487a374c-7313-4fef-898d-a3b27552a4e5 service neutron] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:07:29.450199 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee4d542-23f2-4ed9-99c0-70f2ef534468 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c277338a7f814cc6b01c5419775031bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e30ccf0c1204727b786876c141dbcd2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:07:29.451079 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee4d542-23f2-4ed9-99c0-70f2ef534468 service cinder] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:07:29.451510 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee4d542-23f2-4ed9-99c0-70f2ef534468 service cinder] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:07:29.451927 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee4d542-23f2-4ed9-99c0-70f2ef534468 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:07:29.464716 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee4d542-23f2-4ed9-99c0-70f2ef534468 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:07:29.467007 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee4d542-23f2-4ed9-99c0-70f2ef534468 service cinder] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:07:29.478353 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1212/2406] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 11:07:29 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 11:07:31.010799 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a4b4d3-7db1-4299-adeb-8c0670c7d026 None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:07:31.011307 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a4b4d3-7db1-4299-adeb-8c0670c7d026 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:07:31.011530 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a4b4d3-7db1-4299-adeb-8c0670c7d026 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:07:31.053159 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-c8a4b4d3-7db1-4299-adeb-8c0670c7d026 None None] MFA Rules not processed for user `c3aaf776da3a4bb0853946c195112f40`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 11:07:31.055552 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-c8a4b4d3-7db1-4299-adeb-8c0670c7d026 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:07:31.067444 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1195/2407] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 11:07:31 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 11:07:33.441083 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7147a5d4-da9e-424c-9283-95ee772b377f None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:07:33.442056 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7147a5d4-da9e-424c-9283-95ee772b377f None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:07:33.442056 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7147a5d4-da9e-424c-9283-95ee772b377f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:07:33.461966 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-7147a5d4-da9e-424c-9283-95ee772b377f None None] MFA Rules not processed for user `50be12ccbd274a8792615e9dd10b61ca`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 11:07:33.463219 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-7147a5d4-da9e-424c-9283-95ee772b377f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:07:33.525508 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1213/2408] 10.222.0.40 () {58 vars in 1009 bytes} [Fri Aug 26 11:07:33 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 11:07:33.539117 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a4b4d3-7db1-4299-adeb-8c0670c7d026 None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:07:33.540697 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-c8a4b4d3-7db1-4299-adeb-8c0670c7d026 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:07:33.631522 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a4b4d3-7db1-4299-adeb-8c0670c7d026 None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:07:33.636792 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca4f9ae-dcd8-4267-b56a-0c165c25ab4d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6529bea296c9482a973ced5fe9f2212f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e30ccf0c1204727b786876c141dbcd2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:07:33.637563 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca4f9ae-dcd8-4267-b56a-0c165c25ab4d service swift] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:07:33.637838 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca4f9ae-dcd8-4267-b56a-0c165c25ab4d service swift] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:07:33.638141 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca4f9ae-dcd8-4267-b56a-0c165c25ab4d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:07:33.647658 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca4f9ae-dcd8-4267-b56a-0c165c25ab4d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:07:33.649272 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca4f9ae-dcd8-4267-b56a-0c165c25ab4d service swift] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:07:33.661587 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1196/2409] 10.222.0.40 () {60 vars in 1384 bytes} [Fri Aug 26 11:07:33 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 26 11:07:43.518175 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=88 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 11:07:43.655168 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=41 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 11:07:45.392503 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7147a5d4-da9e-424c-9283-95ee772b377f None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:07:45.394409 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-7147a5d4-da9e-424c-9283-95ee772b377f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:07:45.483403 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7147a5d4-da9e-424c-9283-95ee772b377f None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:07:45.490904 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a820b48a-cac0-4192-992a-db877a91232a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f29c5dfed31f42a9b5cbc0cd72e9fdc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e30ccf0c1204727b786876c141dbcd2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:07:45.491555 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a820b48a-cac0-4192-992a-db877a91232a service placement] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:07:45.491838 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a820b48a-cac0-4192-992a-db877a91232a service placement] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:07:45.492112 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a820b48a-cac0-4192-992a-db877a91232a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:07:45.494240 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-a820b48a-cac0-4192-992a-db877a91232a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:07:45.508555 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a820b48a-cac0-4192-992a-db877a91232a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:07:45.510769 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a820b48a-cac0-4192-992a-db877a91232a service placement] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:07:45.523749 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1214/2410] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 11:07:45 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 11:07:55.518424 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=32 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 11:07:56.172861 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e038ee0-6424-4f41-9077-dba7098f0b12 None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:07:56.173351 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e038ee0-6424-4f41-9077-dba7098f0b12 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:07:56.173444 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e038ee0-6424-4f41-9077-dba7098f0b12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:07:56.198085 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-4e038ee0-6424-4f41-9077-dba7098f0b12 None None] MFA Rules not processed for user `50be12ccbd274a8792615e9dd10b61ca`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 11:07:56.199362 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-4e038ee0-6424-4f41-9077-dba7098f0b12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:07:56.209422 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1197/2411] 10.222.0.40 () {58 vars in 967 bytes} [Fri Aug 26 11:07:56 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 11:07:56.233763 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a820b48a-cac0-4192-992a-db877a91232a service placement] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:07:56.252316 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a820b48a-cac0-4192-992a-db877a91232a service placement] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:07:56.259988 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79908895-252a-42f2-8776-dc3470296af3 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6529bea296c9482a973ced5fe9f2212f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e30ccf0c1204727b786876c141dbcd2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:07:56.260692 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79908895-252a-42f2-8776-dc3470296af3 service swift] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:07:56.261009 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79908895-252a-42f2-8776-dc3470296af3 service swift] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:07:56.261321 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79908895-252a-42f2-8776-dc3470296af3 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:07:56.275969 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79908895-252a-42f2-8776-dc3470296af3 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:07:56.277600 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79908895-252a-42f2-8776-dc3470296af3 service swift] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:07:56.286691 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1215/2412] 10.222.0.40 () {60 vars in 1384 bytes} [Fri Aug 26 11:07:56 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 26 11:08:06.190868 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=3 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 11:08:06.283052 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=6 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 11:08:26.078616 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3943164b-c49b-4675-bd9a-7faf7b9dfc9c None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:08:26.079199 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3943164b-c49b-4675-bd9a-7faf7b9dfc9c None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:08:26.079199 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3943164b-c49b-4675-bd9a-7faf7b9dfc9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:08:26.099114 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-3943164b-c49b-4675-bd9a-7faf7b9dfc9c None None] MFA Rules not processed for user `d6dbbabdd88c4c088bc91e6c43531318`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 11:08:26.100304 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-3943164b-c49b-4675-bd9a-7faf7b9dfc9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:08:26.160456 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1198/2413] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 11:08:26 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 11:08:36.155034 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=25 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 11:08:59.894281 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f42280f-6a68-439e-ae46-3ad590076d30 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:08:59.895514 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f42280f-6a68-439e-ae46-3ad590076d30 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:08:59.895514 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f42280f-6a68-439e-ae46-3ad590076d30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:08:59.924351 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-5f42280f-6a68-439e-ae46-3ad590076d30 None None] MFA Rules not processed for user `c3aaf776da3a4bb0853946c195112f40`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 11:08:59.926430 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-5f42280f-6a68-439e-ae46-3ad590076d30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:08:59.939551 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1216/2414] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 11:08:59 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 11:09:09.916828 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=3 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 11:09:14.310116 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4818a4f9-b6e4-4e4a-bbc1-673c9a4ddb3d None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:09:14.310753 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4818a4f9-b6e4-4e4a-bbc1-673c9a4ddb3d None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:09:14.310753 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4818a4f9-b6e4-4e4a-bbc1-673c9a4ddb3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:09:14.335294 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-4818a4f9-b6e4-4e4a-bbc1-673c9a4ddb3d None None] MFA Rules not processed for user `aedac3f62dd94f87b968f6c71d68a0a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 11:09:14.336432 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-4818a4f9-b6e4-4e4a-bbc1-673c9a4ddb3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:09:14.407892 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1199/2415] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 11:09:14 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 11:09:24.399987 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=25 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 11:09:32.117245 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f42280f-6a68-439e-ae46-3ad590076d30 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:09:32.136013 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f42280f-6a68-439e-ae46-3ad590076d30 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:09:32.140372 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d754af5f-2e11-49bd-a9c1-4a3ee1e53929 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91da2acd2534423091537e1ba068d2b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e30ccf0c1204727b786876c141dbcd2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:09:32.141815 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d754af5f-2e11-49bd-a9c1-4a3ee1e53929 service neutron] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:09:32.142446 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d754af5f-2e11-49bd-a9c1-4a3ee1e53929 service neutron] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:09:32.142446 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d754af5f-2e11-49bd-a9c1-4a3ee1e53929 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:09:32.144278 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-d754af5f-2e11-49bd-a9c1-4a3ee1e53929 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:09:32.166293 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d754af5f-2e11-49bd-a9c1-4a3ee1e53929 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:09:32.169070 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d754af5f-2e11-49bd-a9c1-4a3ee1e53929 service neutron] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:09:32.183534 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1217/2416] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 11:09:32 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 67 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 11:09:33.751585 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4818a4f9-b6e4-4e4a-bbc1-673c9a4ddb3d None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:09:33.767103 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4818a4f9-b6e4-4e4a-bbc1-673c9a4ddb3d None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:09:33.771230 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c8d470-6d66-4094-929a-ba13fb9f8de1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91da2acd2534423091537e1ba068d2b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e30ccf0c1204727b786876c141dbcd2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:09:33.771939 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c8d470-6d66-4094-929a-ba13fb9f8de1 service neutron] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:09:33.772220 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c8d470-6d66-4094-929a-ba13fb9f8de1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:09:33.772638 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c8d470-6d66-4094-929a-ba13fb9f8de1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:09:33.774751 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-49c8d470-6d66-4094-929a-ba13fb9f8de1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:09:33.796099 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c8d470-6d66-4094-929a-ba13fb9f8de1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:09:33.797887 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c8d470-6d66-4094-929a-ba13fb9f8de1 service neutron] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:09:33.818727 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1200/2417] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 11:09:33 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 68 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 11:09:42.177221 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=6 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 11:09:43.799596 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d754af5f-2e11-49bd-a9c1-4a3ee1e53929 service neutron] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:09:43.804865 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=6 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 11:09:43.810029 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d754af5f-2e11-49bd-a9c1-4a3ee1e53929 service neutron] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:09:43.815461 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1142bcd0-db74-4e1f-a73d-bdf53fa86898 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91da2acd2534423091537e1ba068d2b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e30ccf0c1204727b786876c141dbcd2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:09:43.816151 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1142bcd0-db74-4e1f-a73d-bdf53fa86898 service neutron] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:09:43.816427 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1142bcd0-db74-4e1f-a73d-bdf53fa86898 service neutron] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:09:43.816813 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1142bcd0-db74-4e1f-a73d-bdf53fa86898 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:09:43.818884 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-1142bcd0-db74-4e1f-a73d-bdf53fa86898 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:09:43.833590 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1142bcd0-db74-4e1f-a73d-bdf53fa86898 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:09:43.835141 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1142bcd0-db74-4e1f-a73d-bdf53fa86898 service neutron] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:09:43.847230 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1218/2418] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 11:09:43 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 11:09:44.591208 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd11e89-b80c-4552-8ccd-c4d4aff80a57 None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:09:44.591996 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd11e89-b80c-4552-8ccd-c4d4aff80a57 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:09:44.592433 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd11e89-b80c-4552-8ccd-c4d4aff80a57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:09:44.625404 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-0cd11e89-b80c-4552-8ccd-c4d4aff80a57 None None] MFA Rules not processed for user `50be12ccbd274a8792615e9dd10b61ca`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 11:09:44.627542 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-0cd11e89-b80c-4552-8ccd-c4d4aff80a57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:09:44.639158 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1201/2419] 10.222.0.40 () {58 vars in 967 bytes} [Fri Aug 26 11:09:44 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 11:09:44.708597 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1142bcd0-db74-4e1f-a73d-bdf53fa86898 service neutron] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:09:44.722105 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1142bcd0-db74-4e1f-a73d-bdf53fa86898 service neutron] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:09:44.728063 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2dd9c47-249a-4ad3-8d43-42d850e26edc service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6529bea296c9482a973ced5fe9f2212f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e30ccf0c1204727b786876c141dbcd2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:09:44.729609 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2dd9c47-249a-4ad3-8d43-42d850e26edc service swift] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:09:44.730097 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2dd9c47-249a-4ad3-8d43-42d850e26edc service swift] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:09:44.731063 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2dd9c47-249a-4ad3-8d43-42d850e26edc service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:09:44.745485 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2dd9c47-249a-4ad3-8d43-42d850e26edc service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:09:44.750067 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2dd9c47-249a-4ad3-8d43-42d850e26edc service swift] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:09:44.763298 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1219/2420] 10.222.0.40 () {60 vars in 1384 bytes} [Fri Aug 26 11:09:44 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 26 11:09:52.251024 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd11e89-b80c-4552-8ccd-c4d4aff80a57 None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:09:52.253286 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-0cd11e89-b80c-4552-8ccd-c4d4aff80a57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:09:52.269722 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd11e89-b80c-4552-8ccd-c4d4aff80a57 None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:09:52.274634 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad60f343-033f-4eeb-bdf5-042a284bc6c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:09:52.275430 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad60f343-033f-4eeb-bdf5-042a284bc6c2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:09:52.275767 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad60f343-033f-4eeb-bdf5-042a284bc6c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:09:52.276112 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad60f343-033f-4eeb-bdf5-042a284bc6c2 admin admin] PATH_INFO: `/v3/users/be8fb67568674d0b91424e2a16407539` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:09:52.277374 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad60f343-033f-4eeb-bdf5-042a284bc6c2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=be8fb67568674d0b91424e2a16407539)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:09:52.279409 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad60f343-033f-4eeb-bdf5-042a284bc6c2 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:09:52.349671 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-ad60f343-033f-4eeb-bdf5-042a284bc6c2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'be8fb67568674d0b91424e2a16407539'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:09:52.358295 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-ad60f343-033f-4eeb-bdf5-042a284bc6c2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'be8fb67568674d0b91424e2a16407539'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:09:52.359849 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-ad60f343-033f-4eeb-bdf5-042a284bc6c2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'be8fb67568674d0b91424e2a16407539'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:09:52.376987 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-ad60f343-033f-4eeb-bdf5-042a284bc6c2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'be8fb67568674d0b91424e2a16407539'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:09:52.378099 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-ad60f343-033f-4eeb-bdf5-042a284bc6c2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'be8fb67568674d0b91424e2a16407539'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:09:52.395024 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1202/2421] 10.222.0.40 () {58 vars in 1224 bytes} [Fri Aug 26 11:09:52 2022] DELETE /identity/v3/users/be8fb67568674d0b91424e2a16407539 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 26 11:09:52.614948 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2dd9c47-249a-4ad3-8d43-42d850e26edc service swift] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:09:52.651730 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2dd9c47-249a-4ad3-8d43-42d850e26edc service swift] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:09:52.728899 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2cd46ad-3cff-480f-815f-a17473135e8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:09:52.729547 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cd46ad-3cff-480f-815f-a17473135e8c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:09:52.729825 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cd46ad-3cff-480f-815f-a17473135e8c admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:09:52.730783 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cd46ad-3cff-480f-815f-a17473135e8c admin admin] PATH_INFO: `/v3/projects/9483fc8d1764471b99c49e4ee46c0b12` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:09:52.732357 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2cd46ad-3cff-480f-815f-a17473135e8c admin admin] RBAC: Authorizing `identity:delete_project(project_id=9483fc8d1764471b99c49e4ee46c0b12)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:09:52.735533 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2cd46ad-3cff-480f-815f-a17473135e8c admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:09:52.812060 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-d2cd46ad-3cff-480f-815f-a17473135e8c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9483fc8d1764471b99c49e4ee46c0b12', 'request_id': 'req-d2cd46ad-3cff-480f-815f-a17473135e8c'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:09:52.819118 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-d2cd46ad-3cff-480f-815f-a17473135e8c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9483fc8d1764471b99c49e4ee46c0b12', 'request_id': 'req-d2cd46ad-3cff-480f-815f-a17473135e8c'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:09:52.826749 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-d2cd46ad-3cff-480f-815f-a17473135e8c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9483fc8d1764471b99c49e4ee46c0b12', 'request_id': 'req-d2cd46ad-3cff-480f-815f-a17473135e8c'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:09:52.853728 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-d2cd46ad-3cff-480f-815f-a17473135e8c admin admin] The token cache is being invalidate because project 9483fc8d1764471b99c49e4ee46c0b12 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81424) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 11:09:52.854255 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-d2cd46ad-3cff-480f-815f-a17473135e8c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:09:52.858206 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-d2cd46ad-3cff-480f-815f-a17473135e8c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:09:52.861387 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1220/2422] 10.222.0.40 () {58 vars in 1233 bytes} [Fri Aug 26 11:09:52 2022] DELETE /identity/v3/projects/9483fc8d1764471b99c49e4ee46c0b12 => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 11:09:52.867361 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad60f343-033f-4eeb-bdf5-042a284bc6c2 admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:09:52.869238 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-ad60f343-033f-4eeb-bdf5-042a284bc6c2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:09:52.905302 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad60f343-033f-4eeb-bdf5-042a284bc6c2 admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:09:52.975566 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510b80de-9168-4874-bbe2-b1ad91c3ff40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:09:52.976233 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510b80de-9168-4874-bbe2-b1ad91c3ff40 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:09:52.976550 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510b80de-9168-4874-bbe2-b1ad91c3ff40 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:09:52.976816 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510b80de-9168-4874-bbe2-b1ad91c3ff40 admin admin] PATH_INFO: `/v3/users/6fbc95685c2444668c5ef13fe648daed` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:09:52.978116 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-510b80de-9168-4874-bbe2-b1ad91c3ff40 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6fbc95685c2444668c5ef13fe648daed)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:09:52.980267 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-510b80de-9168-4874-bbe2-b1ad91c3ff40 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:09:53.033846 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-510b80de-9168-4874-bbe2-b1ad91c3ff40 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6fbc95685c2444668c5ef13fe648daed'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:09:53.041875 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-510b80de-9168-4874-bbe2-b1ad91c3ff40 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6fbc95685c2444668c5ef13fe648daed'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:09:53.042971 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-510b80de-9168-4874-bbe2-b1ad91c3ff40 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6fbc95685c2444668c5ef13fe648daed'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:09:53.051874 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=81,keystone:DELETE=18,keystone:INSERT=1 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 11:09:53.058110 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-510b80de-9168-4874-bbe2-b1ad91c3ff40 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6fbc95685c2444668c5ef13fe648daed'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:09:53.058920 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-510b80de-9168-4874-bbe2-b1ad91c3ff40 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6fbc95685c2444668c5ef13fe648daed'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:09:53.073805 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1203/2423] 10.222.0.40 () {58 vars in 1224 bytes} [Fri Aug 26 11:09:52 2022] DELETE /identity/v3/users/6fbc95685c2444668c5ef13fe648daed => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 11:09:53.297294 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2cd46ad-3cff-480f-815f-a17473135e8c admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:09:53.298965 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-d2cd46ad-3cff-480f-815f-a17473135e8c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:09:53.316016 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2cd46ad-3cff-480f-815f-a17473135e8c admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:09:53.382578 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7e8daf-3c2f-4253-a0e4-8f79a76151d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:09:53.383222 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7e8daf-3c2f-4253-a0e4-8f79a76151d6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:09:53.383501 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7e8daf-3c2f-4253-a0e4-8f79a76151d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:09:53.383779 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7e8daf-3c2f-4253-a0e4-8f79a76151d6 admin admin] PATH_INFO: `/v3/projects/7c297d01bde64d81a908e764a8c9f03a` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:09:53.384923 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e7e8daf-3c2f-4253-a0e4-8f79a76151d6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7c297d01bde64d81a908e764a8c9f03a)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:09:53.387151 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e7e8daf-3c2f-4253-a0e4-8f79a76151d6 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:09:53.394281 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 11:09:53.437418 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-5e7e8daf-3c2f-4253-a0e4-8f79a76151d6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7c297d01bde64d81a908e764a8c9f03a', 'request_id': 'req-5e7e8daf-3c2f-4253-a0e4-8f79a76151d6'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:09:53.443573 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-5e7e8daf-3c2f-4253-a0e4-8f79a76151d6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7c297d01bde64d81a908e764a8c9f03a', 'request_id': 'req-5e7e8daf-3c2f-4253-a0e4-8f79a76151d6'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:09:53.449763 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-5e7e8daf-3c2f-4253-a0e4-8f79a76151d6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7c297d01bde64d81a908e764a8c9f03a', 'request_id': 'req-5e7e8daf-3c2f-4253-a0e4-8f79a76151d6'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:09:53.463934 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-5e7e8daf-3c2f-4253-a0e4-8f79a76151d6 admin admin] The token cache is being invalidate because project 7c297d01bde64d81a908e764a8c9f03a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81424) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 11:09:53.464309 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-5e7e8daf-3c2f-4253-a0e4-8f79a76151d6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:09:53.465081 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-5e7e8daf-3c2f-4253-a0e4-8f79a76151d6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:09:53.470200 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1221/2424] 10.222.0.40 () {58 vars in 1233 bytes} [Fri Aug 26 11:09:53 2022] DELETE /identity/v3/projects/7c297d01bde64d81a908e764a8c9f03a => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 11:09:53.472195 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fd1bd6-e455-45b8-a51f-2aead1b1b062 None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:09:53.472641 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fd1bd6-e455-45b8-a51f-2aead1b1b062 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:09:53.472747 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fd1bd6-e455-45b8-a51f-2aead1b1b062 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:09:53.493189 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-47fd1bd6-e455-45b8-a51f-2aead1b1b062 None None] MFA Rules not processed for user `ca155a48e2024c32af3fd2e19e5fac11`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 11:09:53.494563 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-47fd1bd6-e455-45b8-a51f-2aead1b1b062 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:09:53.559595 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1204/2425] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 11:09:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 11:09:53.567530 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae233a6-761c-46b9-ac4e-7f09033eca78 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:09:53.567818 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae233a6-761c-46b9-ac4e-7f09033eca78 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:09:53.568098 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae233a6-761c-46b9-ac4e-7f09033eca78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:09:53.589876 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-9ae233a6-761c-46b9-ac4e-7f09033eca78 None None] MFA Rules not processed for user `ca155a48e2024c32af3fd2e19e5fac11`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 11:09:53.591336 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-9ae233a6-761c-46b9-ac4e-7f09033eca78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:09:53.601000 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1222/2426] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 11:09:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 11:09:53.606129 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47fd1bd6-e455-45b8-a51f-2aead1b1b062 None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:09:53.616949 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47fd1bd6-e455-45b8-a51f-2aead1b1b062 None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:09:53.622901 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957d250c-5ed7-44e0-b83d-3d59023593f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:09:53.623516 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957d250c-5ed7-44e0-b83d-3d59023593f9 admin admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:09:53.623793 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957d250c-5ed7-44e0-b83d-3d59023593f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:09:53.624060 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957d250c-5ed7-44e0-b83d-3d59023593f9 admin admin] PATH_INFO: `/v3/domains` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:09:53.625251 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-957d250c-5ed7-44e0-b83d-3d59023593f9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 11:09:53.625733 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-957d250c-5ed7-44e0-b83d-3d59023593f9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:09:53.627879 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-957d250c-5ed7-44e0-b83d-3d59023593f9 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:09:53.641692 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1205/2427] 10.222.0.40 () {58 vars in 1153 bytes} [Fri Aug 26 11:09:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 11:09:53.646589 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae233a6-761c-46b9-ac4e-7f09033eca78 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:09:53.658323 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae233a6-761c-46b9-ac4e-7f09033eca78 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:09:53.663041 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d291c3e-4b19-421b-bf7f-a1c6a3e3ba76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:09:53.663645 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d291c3e-4b19-421b-bf7f-a1c6a3e3ba76 admin admin] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:09:53.663920 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d291c3e-4b19-421b-bf7f-a1c6a3e3ba76 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:09:53.664188 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d291c3e-4b19-421b-bf7f-a1c6a3e3ba76 admin admin] PATH_INFO: `/v3/projects` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:09:53.664818 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d291c3e-4b19-421b-bf7f-a1c6a3e3ba76 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:09:53.666764 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d291c3e-4b19-421b-bf7f-a1c6a3e3ba76 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:09:53.696349 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1223/2428] 10.222.0.40 () {60 vars in 1153 bytes} [Fri Aug 26 11:09:53 2022] POST /identity/v3/projects => generated 377 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 11:09:53.700573 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957d250c-5ed7-44e0-b83d-3d59023593f9 admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:09:53.711628 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957d250c-5ed7-44e0-b83d-3d59023593f9 admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:09:53.769762 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f88ddc0-7df7-4c43-8ac3-f3779ec3cbcf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:09:53.770396 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f88ddc0-7df7-4c43-8ac3-f3779ec3cbcf admin admin] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:09:53.770678 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f88ddc0-7df7-4c43-8ac3-f3779ec3cbcf admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:09:53.770949 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f88ddc0-7df7-4c43-8ac3-f3779ec3cbcf admin admin] PATH_INFO: `/v3/users` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:09:53.771585 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f88ddc0-7df7-4c43-8ac3-f3779ec3cbcf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:09:53.773481 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f88ddc0-7df7-4c43-8ac3-f3779ec3cbcf admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:09:53.795518 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1206/2429] 10.222.0.40 () {60 vars in 1144 bytes} [Fri Aug 26 11:09:53 2022] POST /identity/v3/users => generated 289 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 11:09:53.799948 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d291c3e-4b19-421b-bf7f-a1c6a3e3ba76 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:09:53.810526 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d291c3e-4b19-421b-bf7f-a1c6a3e3ba76 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:09:53.816294 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414d911a-4326-45aa-b6a8-c10e605dcaf5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:09:53.816897 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414d911a-4326-45aa-b6a8-c10e605dcaf5 admin admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:09:53.817170 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414d911a-4326-45aa-b6a8-c10e605dcaf5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:09:53.817437 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414d911a-4326-45aa-b6a8-c10e605dcaf5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:09:53.818118 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-414d911a-4326-45aa-b6a8-c10e605dcaf5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:09:53.819711 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-414d911a-4326-45aa-b6a8-c10e605dcaf5 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:09:53.830811 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1224/2430] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 26 11:09:53 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 11:09:53.834900 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f88ddc0-7df7-4c43-8ac3-f3779ec3cbcf admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:09:53.845509 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f88ddc0-7df7-4c43-8ac3-f3779ec3cbcf admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:09:53.850131 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807ca0f7-bb63-492b-913b-e9522c343f5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:09:53.850726 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807ca0f7-bb63-492b-913b-e9522c343f5a admin admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:09:53.850999 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807ca0f7-bb63-492b-913b-e9522c343f5a admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:09:53.851267 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807ca0f7-bb63-492b-913b-e9522c343f5a admin admin] PATH_INFO: `/v3/roles` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:09:53.851901 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-807ca0f7-bb63-492b-913b-e9522c343f5a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:09:53.853689 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-807ca0f7-bb63-492b-913b-e9522c343f5a admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:09:53.865159 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1207/2431] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 26 11:09:53 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 26 11:09:53.869578 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414d911a-4326-45aa-b6a8-c10e605dcaf5 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:09:53.881869 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414d911a-4326-45aa-b6a8-c10e605dcaf5 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:09:53.886860 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8b72ff-701f-488c-96a2-1472f28c4d5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:09:53.887523 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8b72ff-701f-488c-96a2-1472f28c4d5f admin admin] REQUEST_METHOD: `PUT` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:09:53.887797 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8b72ff-701f-488c-96a2-1472f28c4d5f admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:09:53.888069 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8b72ff-701f-488c-96a2-1472f28c4d5f admin admin] PATH_INFO: `/v3/projects/d773f20240a44729b159c57c80e017ef/users/9e8355dc324e49d881b9f0fc5c2923ad/roles/23e174ce743b46eabe618c212691da55` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:09:53.904461 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f8b72ff-701f-488c-96a2-1472f28c4d5f admin admin] RBAC: Authorizing `identity:create_grant(project_id=d773f20240a44729b159c57c80e017ef, user_id=9e8355dc324e49d881b9f0fc5c2923ad, role_id=23e174ce743b46eabe618c212691da55)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:09:53.907628 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f8b72ff-701f-488c-96a2-1472f28c4d5f admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:09:53.928481 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1225/2432] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 26 11:09:53 2022] PUT /identity/v3/projects/d773f20240a44729b159c57c80e017ef/users/9e8355dc324e49d881b9f0fc5c2923ad/roles/23e174ce743b46eabe618c212691da55 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 11:09:53.933475 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ee65be-172b-44f7-8c86-e825cdb3fe70 None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:09:53.933758 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ee65be-172b-44f7-8c86-e825cdb3fe70 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:09:53.934046 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ee65be-172b-44f7-8c86-e825cdb3fe70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:09:53.955648 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-d4ee65be-172b-44f7-8c86-e825cdb3fe70 None None] MFA Rules not processed for user `ca155a48e2024c32af3fd2e19e5fac11`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 11:09:53.956857 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-d4ee65be-172b-44f7-8c86-e825cdb3fe70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:09:54.004142 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=94,keystone:DELETE=2,keystone:INSERT=4 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 11:09:54.022193 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1208/2433] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 11:09:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 11:09:54.032226 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8b72ff-701f-488c-96a2-1472f28c4d5f admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:09:54.034444 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-6f8b72ff-701f-488c-96a2-1472f28c4d5f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:09:54.102138 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8b72ff-701f-488c-96a2-1472f28c4d5f admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:09:54.106065 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27153929-af7e-46b6-be23-df12c3c55cec service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91da2acd2534423091537e1ba068d2b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e30ccf0c1204727b786876c141dbcd2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:09:54.106687 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27153929-af7e-46b6-be23-df12c3c55cec service neutron] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:09:54.106963 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27153929-af7e-46b6-be23-df12c3c55cec service neutron] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:09:54.107233 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27153929-af7e-46b6-be23-df12c3c55cec service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:09:54.117314 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27153929-af7e-46b6-be23-df12c3c55cec service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:09:54.118797 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27153929-af7e-46b6-be23-df12c3c55cec service neutron] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:09:54.129396 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1226/2434] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 11:09:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 26 11:09:54.365305 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ee65be-172b-44f7-8c86-e825cdb3fe70 None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:09:54.376215 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ee65be-172b-44f7-8c86-e825cdb3fe70 None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:09:54.379907 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5babef-9528-4b99-a49d-5faaff4f85f0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91da2acd2534423091537e1ba068d2b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e30ccf0c1204727b786876c141dbcd2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:09:54.380512 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5babef-9528-4b99-a49d-5faaff4f85f0 service neutron] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:09:54.380786 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5babef-9528-4b99-a49d-5faaff4f85f0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:09:54.381052 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5babef-9528-4b99-a49d-5faaff4f85f0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:09:54.382840 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-fd5babef-9528-4b99-a49d-5faaff4f85f0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:09:54.396456 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd5babef-9528-4b99-a49d-5faaff4f85f0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:09:54.397888 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd5babef-9528-4b99-a49d-5faaff4f85f0 service neutron] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:09:54.409824 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1209/2435] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 11:09:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 11:09:59.154830 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27153929-af7e-46b6-be23-df12c3c55cec service neutron] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:09:59.158503 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-27153929-af7e-46b6-be23-df12c3c55cec service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:09:59.175355 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27153929-af7e-46b6-be23-df12c3c55cec service neutron] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:09:59.183344 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb26d44c-54f3-433f-a38d-5448c0eb209d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:09:59.183938 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb26d44c-54f3-433f-a38d-5448c0eb209d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:09:59.184184 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb26d44c-54f3-433f-a38d-5448c0eb209d admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:09:59.184718 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb26d44c-54f3-433f-a38d-5448c0eb209d admin admin] PATH_INFO: `/v3/users/d6dbbabdd88c4c088bc91e6c43531318` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:09:59.186150 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb26d44c-54f3-433f-a38d-5448c0eb209d admin admin] RBAC: Authorizing `identity:delete_user(user_id=d6dbbabdd88c4c088bc91e6c43531318)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:09:59.189677 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb26d44c-54f3-433f-a38d-5448c0eb209d admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:09:59.247037 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 11:09:59.286273 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-bb26d44c-54f3-433f-a38d-5448c0eb209d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd6dbbabdd88c4c088bc91e6c43531318'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:09:59.287630 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-bb26d44c-54f3-433f-a38d-5448c0eb209d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd6dbbabdd88c4c088bc91e6c43531318'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:09:59.302026 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-bb26d44c-54f3-433f-a38d-5448c0eb209d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd6dbbabdd88c4c088bc91e6c43531318'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:09:59.303075 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-bb26d44c-54f3-433f-a38d-5448c0eb209d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd6dbbabdd88c4c088bc91e6c43531318'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:09:59.319084 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-bb26d44c-54f3-433f-a38d-5448c0eb209d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd6dbbabdd88c4c088bc91e6c43531318'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:09:59.341959 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1227/2436] 10.222.0.40 () {58 vars in 1224 bytes} [Fri Aug 26 11:09:59 2022] DELETE /identity/v3/users/d6dbbabdd88c4c088bc91e6c43531318 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 11:09:59.570272 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83834efc-eb89-4964-b39d-1f99980e9b4c None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:09:59.571857 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83834efc-eb89-4964-b39d-1f99980e9b4c None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:09:59.572282 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83834efc-eb89-4964-b39d-1f99980e9b4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:09:59.607143 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-83834efc-eb89-4964-b39d-1f99980e9b4c None None] MFA Rules not processed for user `9e8355dc324e49d881b9f0fc5c2923ad`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 11:09:59.615807 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb26d44c-54f3-433f-a38d-5448c0eb209d admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:09:59.615906 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-83834efc-eb89-4964-b39d-1f99980e9b4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:09:59.617009 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-bb26d44c-54f3-433f-a38d-5448c0eb209d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:09:59.746361 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1210/2437] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 11:09:59 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 11:09:59.748900 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83834efc-eb89-4964-b39d-1f99980e9b4c None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:09:59.751888 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-83834efc-eb89-4964-b39d-1f99980e9b4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:09:59.754314 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb26d44c-54f3-433f-a38d-5448c0eb209d admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:09:59.759226 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb05212e-71f9-48ad-bcfc-ed4a6ad6ae59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:09:59.759932 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb05212e-71f9-48ad-bcfc-ed4a6ad6ae59 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:09:59.760218 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb05212e-71f9-48ad-bcfc-ed4a6ad6ae59 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:09:59.760495 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb05212e-71f9-48ad-bcfc-ed4a6ad6ae59 admin admin] PATH_INFO: `/v3/projects/7db3b983562e49cfbb77c9e64b6e6cb9` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:09:59.761847 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb05212e-71f9-48ad-bcfc-ed4a6ad6ae59 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7db3b983562e49cfbb77c9e64b6e6cb9)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:09:59.763843 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb05212e-71f9-48ad-bcfc-ed4a6ad6ae59 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:09:59.772502 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83834efc-eb89-4964-b39d-1f99980e9b4c None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:09:59.779306 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9808c173-2cfe-4426-8a78-3821f2d3f510 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:09:59.779975 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9808c173-2cfe-4426-8a78-3821f2d3f510 admin admin] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:09:59.780273 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9808c173-2cfe-4426-8a78-3821f2d3f510 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:09:59.780601 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9808c173-2cfe-4426-8a78-3821f2d3f510 admin admin] PATH_INFO: `/v3/projects` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:09:59.781744 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9808c173-2cfe-4426-8a78-3821f2d3f510 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:09:59.784115 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9808c173-2cfe-4426-8a78-3821f2d3f510 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:09:59.825048 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1211/2438] 10.222.0.40 () {60 vars in 1153 bytes} [Fri Aug 26 11:09:59 2022] POST /identity/v3/projects => generated 377 bytes in 73 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 11:09:59.826685 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9808c173-2cfe-4426-8a78-3821f2d3f510 admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:09:59.827068 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-eb05212e-71f9-48ad-bcfc-ed4a6ad6ae59 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7db3b983562e49cfbb77c9e64b6e6cb9', 'request_id': 'req-eb05212e-71f9-48ad-bcfc-ed4a6ad6ae59'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:09:59.846258 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-eb05212e-71f9-48ad-bcfc-ed4a6ad6ae59 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7db3b983562e49cfbb77c9e64b6e6cb9', 'request_id': 'req-eb05212e-71f9-48ad-bcfc-ed4a6ad6ae59'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:09:59.868104 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-eb05212e-71f9-48ad-bcfc-ed4a6ad6ae59 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7db3b983562e49cfbb77c9e64b6e6cb9', 'request_id': 'req-eb05212e-71f9-48ad-bcfc-ed4a6ad6ae59'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:09:59.897344 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-eb05212e-71f9-48ad-bcfc-ed4a6ad6ae59 admin admin] The token cache is being invalidate because project 7db3b983562e49cfbb77c9e64b6e6cb9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81424) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 11:09:59.897709 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-eb05212e-71f9-48ad-bcfc-ed4a6ad6ae59 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:09:59.898637 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-eb05212e-71f9-48ad-bcfc-ed4a6ad6ae59 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:09:59.902357 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1228/2439] 10.222.0.40 () {58 vars in 1233 bytes} [Fri Aug 26 11:09:59 2022] DELETE /identity/v3/projects/7db3b983562e49cfbb77c9e64b6e6cb9 => generated 0 bytes in 290 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 11:09:59.908568 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9acbf1-27ca-481b-a0f0-c296026c55e2 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:09:59.908948 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9acbf1-27ca-481b-a0f0-c296026c55e2 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:09:59.909453 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9acbf1-27ca-481b-a0f0-c296026c55e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:09:59.948264 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-1a9acbf1-27ca-481b-a0f0-c296026c55e2 None None] MFA Rules not processed for user `ca155a48e2024c32af3fd2e19e5fac11`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 11:09:59.949554 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-1a9acbf1-27ca-481b-a0f0-c296026c55e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:09:59.957589 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9808c173-2cfe-4426-8a78-3821f2d3f510 admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:09:59.963121 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fdd2040-dbab-4f9f-9230-07e06307e13a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:09:59.963940 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fdd2040-dbab-4f9f-9230-07e06307e13a admin admin] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:09:59.964219 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fdd2040-dbab-4f9f-9230-07e06307e13a admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:09:59.964488 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fdd2040-dbab-4f9f-9230-07e06307e13a admin admin] PATH_INFO: `/v3/users` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:09:59.965130 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fdd2040-dbab-4f9f-9230-07e06307e13a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:09:59.973892 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1229/2440] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 11:09:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 59 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 11:09:59.974823 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fdd2040-dbab-4f9f-9230-07e06307e13a admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:09:59.975571 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231e8562-de38-4205-bd91-36bf8d4b74c8 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:09:59.975867 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231e8562-de38-4205-bd91-36bf8d4b74c8 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:09:59.976144 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231e8562-de38-4205-bd91-36bf8d4b74c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:10:00.007864 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-231e8562-de38-4205-bd91-36bf8d4b74c8 None None] MFA Rules not processed for user `ca155a48e2024c32af3fd2e19e5fac11`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 11:10:00.018952 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-231e8562-de38-4205-bd91-36bf8d4b74c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:10:00.025426 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1212/2441] 10.222.0.40 () {60 vars in 1144 bytes} [Fri Aug 26 11:09:59 2022] POST /identity/v3/users => generated 295 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 11:10:00.029127 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fdd2040-dbab-4f9f-9230-07e06307e13a admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:10:00.034062 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-6fdd2040-dbab-4f9f-9230-07e06307e13a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:10:00.034300 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1230/2442] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 11:09:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 11:10:00.039146 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231e8562-de38-4205-bd91-36bf8d4b74c8 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:10:00.047431 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fdd2040-dbab-4f9f-9230-07e06307e13a admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:10:00.052698 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec04ab5e-ab7c-40e1-9bbd-696d05f1d20c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:10:00.054004 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec04ab5e-ab7c-40e1-9bbd-696d05f1d20c admin admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:10:00.054004 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec04ab5e-ab7c-40e1-9bbd-696d05f1d20c admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:10:00.054004 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec04ab5e-ab7c-40e1-9bbd-696d05f1d20c admin admin] PATH_INFO: `/v3/roles` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:10:00.054551 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec04ab5e-ab7c-40e1-9bbd-696d05f1d20c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:10:00.056739 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec04ab5e-ab7c-40e1-9bbd-696d05f1d20c admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:10:00.068739 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231e8562-de38-4205-bd91-36bf8d4b74c8 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:10:00.075447 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1213/2443] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 26 11:10:00 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 11:10:00.079861 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec04ab5e-ab7c-40e1-9bbd-696d05f1d20c admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:10:00.082520 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e9c4c8-7846-4d74-9423-e0c35688790a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:10:00.083800 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e9c4c8-7846-4d74-9423-e0c35688790a admin admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:10:00.083800 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e9c4c8-7846-4d74-9423-e0c35688790a admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:10:00.084041 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e9c4c8-7846-4d74-9423-e0c35688790a admin admin] PATH_INFO: `/v3/domains` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:10:00.084962 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e9c4c8-7846-4d74-9423-e0c35688790a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 11:10:00.086221 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e9c4c8-7846-4d74-9423-e0c35688790a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:10:00.089711 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e9c4c8-7846-4d74-9423-e0c35688790a admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:10:00.093742 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec04ab5e-ab7c-40e1-9bbd-696d05f1d20c admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:10:00.099774 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6999ac02-da95-4c61-b440-df535094c056 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:10:00.100585 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6999ac02-da95-4c61-b440-df535094c056 admin admin] REQUEST_METHOD: `PUT` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:10:00.100924 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6999ac02-da95-4c61-b440-df535094c056 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:10:00.101274 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6999ac02-da95-4c61-b440-df535094c056 admin admin] PATH_INFO: `/v3/projects/e187724eb241446f840cb3580c714abf/users/379400ce9edd4388b57f43664a2d62c7/roles/1f97e4602e2e4d6498913a81f9ae8d6a` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:10:00.112904 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1231/2444] 10.222.0.40 () {58 vars in 1153 bytes} [Fri Aug 26 11:10:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 11:10:00.117605 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e9c4c8-7846-4d74-9423-e0c35688790a admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:10:00.127195 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6999ac02-da95-4c61-b440-df535094c056 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e187724eb241446f840cb3580c714abf, user_id=379400ce9edd4388b57f43664a2d62c7, role_id=1f97e4602e2e4d6498913a81f9ae8d6a)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:10:00.130189 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e9c4c8-7846-4d74-9423-e0c35688790a admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:10:00.130734 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6999ac02-da95-4c61-b440-df535094c056 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:10:00.135603 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a4b902-efe2-4362-a6a5-a1389af300d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:10:00.136349 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a4b902-efe2-4362-a6a5-a1389af300d2 admin admin] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:10:00.136703 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a4b902-efe2-4362-a6a5-a1389af300d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:10:00.137044 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a4b902-efe2-4362-a6a5-a1389af300d2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:10:00.137906 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a4b902-efe2-4362-a6a5-a1389af300d2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:10:00.140816 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a4b902-efe2-4362-a6a5-a1389af300d2 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:10:00.148607 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 11:10:00.151109 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 11:10:00.163836 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1214/2445] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 26 11:10:00 2022] PUT /identity/v3/projects/e187724eb241446f840cb3580c714abf/users/379400ce9edd4388b57f43664a2d62c7/roles/1f97e4602e2e4d6498913a81f9ae8d6a => generated 0 bytes in 82 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 11:10:00.166582 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6999ac02-da95-4c61-b440-df535094c056 admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:10:00.181718 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1232/2446] 10.222.0.40 () {60 vars in 1153 bytes} [Fri Aug 26 11:10:00 2022] POST /identity/v3/projects => generated 383 bytes in 65 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 26 11:10:00.188822 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a4b902-efe2-4362-a6a5-a1389af300d2 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:10:00.255992 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6999ac02-da95-4c61-b440-df535094c056 admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:10:00.290971 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a4b902-efe2-4362-a6a5-a1389af300d2 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:10:00.299276 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9df0794-7182-4375-8234-f05baaf6c2b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:10:00.300236 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9df0794-7182-4375-8234-f05baaf6c2b0 admin admin] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:10:00.300692 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9df0794-7182-4375-8234-f05baaf6c2b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:10:00.301141 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9df0794-7182-4375-8234-f05baaf6c2b0 admin admin] PATH_INFO: `/v3/users` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:10:00.302572 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9df0794-7182-4375-8234-f05baaf6c2b0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:10:00.306034 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9df0794-7182-4375-8234-f05baaf6c2b0 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:10:00.338491 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c406b2b-93ae-4471-8ef4-f7c0f31ae77c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:10:00.339074 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c406b2b-93ae-4471-8ef4-f7c0f31ae77c admin admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:10:00.339613 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c406b2b-93ae-4471-8ef4-f7c0f31ae77c admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:10:00.339712 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c406b2b-93ae-4471-8ef4-f7c0f31ae77c admin admin] PATH_INFO: `/v3/roles` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:10:00.340458 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c406b2b-93ae-4471-8ef4-f7c0f31ae77c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:10:00.341146 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1233/2447] 10.222.0.40 () {60 vars in 1144 bytes} [Fri Aug 26 11:10:00 2022] POST /identity/v3/users => generated 292 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 26 11:10:00.342336 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c406b2b-93ae-4471-8ef4-f7c0f31ae77c admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:10:00.347894 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9df0794-7182-4375-8234-f05baaf6c2b0 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:10:00.357293 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1215/2448] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 26 11:10:00 2022] GET /identity/v3/roles => generated 1420 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 11:10:00.359882 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9df0794-7182-4375-8234-f05baaf6c2b0 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:10:00.361842 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c406b2b-93ae-4471-8ef4-f7c0f31ae77c admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:10:00.364901 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adbf9948-72cf-431c-9640-9e75e4ccf1ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:10:00.365533 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbf9948-72cf-431c-9640-9e75e4ccf1ae admin admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:10:00.365853 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbf9948-72cf-431c-9640-9e75e4ccf1ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:10:00.366160 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbf9948-72cf-431c-9640-9e75e4ccf1ae admin admin] PATH_INFO: `/v3/roles` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:10:00.366808 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adbf9948-72cf-431c-9640-9e75e4ccf1ae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:10:00.368545 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adbf9948-72cf-431c-9640-9e75e4ccf1ae admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:10:00.373641 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c406b2b-93ae-4471-8ef4-f7c0f31ae77c admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:10:00.381437 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03cce07-1de2-4334-a9f9-54846b23b3c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:10:00.382856 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1234/2449] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 26 11:10:00 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 26 11:10:00.382983 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03cce07-1de2-4334-a9f9-54846b23b3c2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:10:00.383357 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03cce07-1de2-4334-a9f9-54846b23b3c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:10:00.384009 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03cce07-1de2-4334-a9f9-54846b23b3c2 admin admin] PATH_INFO: `/v3/projects/e187724eb241446f840cb3580c714abf/users/379400ce9edd4388b57f43664a2d62c7/roles/1f97e4602e2e4d6498913a81f9ae8d6a` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:10:00.387763 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f03cce07-1de2-4334-a9f9-54846b23b3c2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e187724eb241446f840cb3580c714abf, user_id=379400ce9edd4388b57f43664a2d62c7, role_id=1f97e4602e2e4d6498913a81f9ae8d6a)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:10:00.390216 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adbf9948-72cf-431c-9640-9e75e4ccf1ae admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:10:00.392281 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f03cce07-1de2-4334-a9f9-54846b23b3c2 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:10:00.408358 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adbf9948-72cf-431c-9640-9e75e4ccf1ae admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:10:00.413074 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1216/2450] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 26 11:10:00 2022] PUT /identity/v3/projects/e187724eb241446f840cb3580c714abf/users/379400ce9edd4388b57f43664a2d62c7/roles/1f97e4602e2e4d6498913a81f9ae8d6a => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 26 11:10:00.540537 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c32d32-89e4-4e7e-bb04-9bf4bb3b27a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:10:00.541362 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c32d32-89e4-4e7e-bb04-9bf4bb3b27a2 admin admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:10:00.542517 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c32d32-89e4-4e7e-bb04-9bf4bb3b27a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:10:00.542939 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c32d32-89e4-4e7e-bb04-9bf4bb3b27a2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:10:00.544432 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c32d32-89e4-4e7e-bb04-9bf4bb3b27a2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:10:00.546964 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c32d32-89e4-4e7e-bb04-9bf4bb3b27a2 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:10:00.572018 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1235/2451] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 26 11:10:00 2022] GET /identity/v3/roles => generated 1420 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 26 11:10:00.574284 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03cce07-1de2-4334-a9f9-54846b23b3c2 admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:10:00.595771 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03cce07-1de2-4334-a9f9-54846b23b3c2 admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:10:00.602724 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cdbf9c0-294a-4958-bfdf-a77979e0930c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:10:00.603404 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cdbf9c0-294a-4958-bfdf-a77979e0930c admin admin] REQUEST_METHOD: `PUT` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:10:00.603675 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cdbf9c0-294a-4958-bfdf-a77979e0930c admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:10:00.604424 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cdbf9c0-294a-4958-bfdf-a77979e0930c admin admin] PATH_INFO: `/v3/projects/f780402b93f94d26b2fceab83c0bf485/users/85716f53e88048af9a8fdcbe4415979c/roles/23e174ce743b46eabe618c212691da55` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:10:00.624009 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cdbf9c0-294a-4958-bfdf-a77979e0930c admin admin] RBAC: Authorizing `identity:create_grant(project_id=f780402b93f94d26b2fceab83c0bf485, user_id=85716f53e88048af9a8fdcbe4415979c, role_id=23e174ce743b46eabe618c212691da55)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:10:00.628415 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cdbf9c0-294a-4958-bfdf-a77979e0930c admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:10:00.654332 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1217/2452] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 26 11:10:00 2022] PUT /identity/v3/projects/f780402b93f94d26b2fceab83c0bf485/users/85716f53e88048af9a8fdcbe4415979c/roles/23e174ce743b46eabe618c212691da55 => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 11:10:00.657052 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87eb5002-f22d-40c5-82e7-01044106d26f None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:10:00.659052 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87eb5002-f22d-40c5-82e7-01044106d26f None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:10:00.659052 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87eb5002-f22d-40c5-82e7-01044106d26f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:10:00.683349 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-87eb5002-f22d-40c5-82e7-01044106d26f None None] MFA Rules not processed for user `ca155a48e2024c32af3fd2e19e5fac11`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 11:10:00.684860 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-87eb5002-f22d-40c5-82e7-01044106d26f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:10:00.807941 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1236/2453] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 11:10:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 11:10:00.820956 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cdbf9c0-294a-4958-bfdf-a77979e0930c admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:10:00.823812 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-5cdbf9c0-294a-4958-bfdf-a77979e0930c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:10:00.924915 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cdbf9c0-294a-4958-bfdf-a77979e0930c admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:10:00.935270 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7db740d-fc1d-4fa3-9759-b237631e5cbb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91da2acd2534423091537e1ba068d2b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e30ccf0c1204727b786876c141dbcd2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:10:00.935270 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7db740d-fc1d-4fa3-9759-b237631e5cbb service neutron] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:10:00.935702 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7db740d-fc1d-4fa3-9759-b237631e5cbb service neutron] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:10:00.936055 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7db740d-fc1d-4fa3-9759-b237631e5cbb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:10:00.964619 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7db740d-fc1d-4fa3-9759-b237631e5cbb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:10:00.970290 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7db740d-fc1d-4fa3-9759-b237631e5cbb service neutron] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:10:01.002091 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1218/2454] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 11:10:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 26 11:10:09.652873 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87eb5002-f22d-40c5-82e7-01044106d26f None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:10:09.654467 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-87eb5002-f22d-40c5-82e7-01044106d26f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:10:09.662800 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 11:10:09.754018 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87eb5002-f22d-40c5-82e7-01044106d26f None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:10:09.760292 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b565699a-02f1-4acf-b6fd-c86b6b1b5a28 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c277338a7f814cc6b01c5419775031bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e30ccf0c1204727b786876c141dbcd2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:10:09.761088 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b565699a-02f1-4acf-b6fd-c86b6b1b5a28 service cinder] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:10:09.761532 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b565699a-02f1-4acf-b6fd-c86b6b1b5a28 service cinder] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:10:09.763123 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b565699a-02f1-4acf-b6fd-c86b6b1b5a28 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:10:09.766272 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-b565699a-02f1-4acf-b6fd-c86b6b1b5a28 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:10:09.827932 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7271b040-e720-4880-a331-1538df319816 None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:10:09.828861 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7271b040-e720-4880-a331-1538df319816 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:10:09.829716 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7271b040-e720-4880-a331-1538df319816 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:10:09.869933 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b565699a-02f1-4acf-b6fd-c86b6b1b5a28 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:10:09.872834 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b565699a-02f1-4acf-b6fd-c86b6b1b5a28 service cinder] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:10:09.883469 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 11:10:09.906659 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1237/2455] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 11:10:09 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 253 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 11:10:09.913910 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-7271b040-e720-4880-a331-1538df319816 None None] MFA Rules not processed for user `379400ce9edd4388b57f43664a2d62c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 11:10:09.915481 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-7271b040-e720-4880-a331-1538df319816 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:10:10.120454 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1219/2456] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 11:10:09 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 292 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 11:10:10.130244 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b565699a-02f1-4acf-b6fd-c86b6b1b5a28 service cinder] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:10:10.131617 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-b565699a-02f1-4acf-b6fd-c86b6b1b5a28 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:10:10.226696 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4689bfd3-7396-4b91-b3b1-880a7f32683d None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:10:10.226696 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4689bfd3-7396-4b91-b3b1-880a7f32683d None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:10:10.226696 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4689bfd3-7396-4b91-b3b1-880a7f32683d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:10:10.278088 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-4689bfd3-7396-4b91-b3b1-880a7f32683d None None] MFA Rules not processed for user `85716f53e88048af9a8fdcbe4415979c`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 11:10:10.283199 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-4689bfd3-7396-4b91-b3b1-880a7f32683d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:10:10.326453 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b565699a-02f1-4acf-b6fd-c86b6b1b5a28 service cinder] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:10:10.334927 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3904cf1-04c3-4d83-b766-7d00e2af4ba1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f4b4a662db049ecacc865ecff10bdb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e30ccf0c1204727b786876c141dbcd2', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:10:10.335766 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3904cf1-04c3-4d83-b766-7d00e2af4ba1 service nova] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:10:10.336097 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3904cf1-04c3-4d83-b766-7d00e2af4ba1 service nova] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:10:10.336372 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3904cf1-04c3-4d83-b766-7d00e2af4ba1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:10:10.338613 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-a3904cf1-04c3-4d83-b766-7d00e2af4ba1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:10:10.427654 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1220/2457] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 11:10:10 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 11:10:10.442520 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 11:10:10.448832 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3904cf1-04c3-4d83-b766-7d00e2af4ba1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:10:10.452255 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3904cf1-04c3-4d83-b766-7d00e2af4ba1 service nova] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:10:10.473072 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1238/2458] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 11:10:10 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 11:10:10.491512 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4689bfd3-7396-4b91-b3b1-880a7f32683d None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:10:10.494662 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-4689bfd3-7396-4b91-b3b1-880a7f32683d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:10:10.520445 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4689bfd3-7396-4b91-b3b1-880a7f32683d None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:10:10.531587 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9edf4eba-3830-4cf1-94a9-80289c77a06a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f4b4a662db049ecacc865ecff10bdb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e30ccf0c1204727b786876c141dbcd2', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:10:10.532325 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9edf4eba-3830-4cf1-94a9-80289c77a06a service nova] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:10:10.532607 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9edf4eba-3830-4cf1-94a9-80289c77a06a service nova] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:10:10.532979 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9edf4eba-3830-4cf1-94a9-80289c77a06a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:10:10.551758 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9edf4eba-3830-4cf1-94a9-80289c77a06a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:10:10.562890 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9edf4eba-3830-4cf1-94a9-80289c77a06a service nova] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:10:10.610485 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1221/2459] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 11:10:10 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 11:10:13.347549 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef1a6d5-bd92-42f0-bc40-ee0aa12e0f30 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:10:13.349671 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef1a6d5-bd92-42f0-bc40-ee0aa12e0f30 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:10:13.350334 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef1a6d5-bd92-42f0-bc40-ee0aa12e0f30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:10:13.390900 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9edf4eba-3830-4cf1-94a9-80289c77a06a service nova] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:10:13.396253 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-9edf4eba-3830-4cf1-94a9-80289c77a06a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:10:13.448241 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9edf4eba-3830-4cf1-94a9-80289c77a06a service nova] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:10:13.455513 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-7ef1a6d5-bd92-42f0-bc40-ee0aa12e0f30 None None] MFA Rules not processed for user `85716f53e88048af9a8fdcbe4415979c`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 11:10:13.463489 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-7ef1a6d5-bd92-42f0-bc40-ee0aa12e0f30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:10:13.470540 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1b787a-b361-4fa0-a228-e269a489a750 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:10:13.471633 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1b787a-b361-4fa0-a228-e269a489a750 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:10:13.472094 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1b787a-b361-4fa0-a228-e269a489a750 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:10:13.472543 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1b787a-b361-4fa0-a228-e269a489a750 admin admin] PATH_INFO: `/v3/users/c3aaf776da3a4bb0853946c195112f40` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:10:13.474094 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e1b787a-b361-4fa0-a228-e269a489a750 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c3aaf776da3a4bb0853946c195112f40)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:10:13.482804 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e1b787a-b361-4fa0-a228-e269a489a750 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:10:13.506007 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1239/2460] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 11:10:13 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 11:10:13.595056 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-0e1b787a-b361-4fa0-a228-e269a489a750 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c3aaf776da3a4bb0853946c195112f40'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:10:13.610955 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-0e1b787a-b361-4fa0-a228-e269a489a750 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c3aaf776da3a4bb0853946c195112f40'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:10:13.613221 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-0e1b787a-b361-4fa0-a228-e269a489a750 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c3aaf776da3a4bb0853946c195112f40'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:10:13.636234 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=92,keystone:DELETE=8 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 11:10:13.652367 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-0e1b787a-b361-4fa0-a228-e269a489a750 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c3aaf776da3a4bb0853946c195112f40'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:10:13.653127 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-0e1b787a-b361-4fa0-a228-e269a489a750 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c3aaf776da3a4bb0853946c195112f40'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:10:13.680637 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1222/2461] 10.222.0.40 () {58 vars in 1224 bytes} [Fri Aug 26 11:10:13 2022] DELETE /identity/v3/users/c3aaf776da3a4bb0853946c195112f40 => generated 0 bytes in 291 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 11:10:14.076037 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef1a6d5-bd92-42f0-bc40-ee0aa12e0f30 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:10:14.106975 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef1a6d5-bd92-42f0-bc40-ee0aa12e0f30 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:10:14.282821 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c912e27-fd46-43ac-bf95-d61057e5d7ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:10:14.282821 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c912e27-fd46-43ac-bf95-d61057e5d7ac admin admin] REQUEST_METHOD: `DELETE` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:10:14.282821 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c912e27-fd46-43ac-bf95-d61057e5d7ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:10:14.282821 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c912e27-fd46-43ac-bf95-d61057e5d7ac admin admin] PATH_INFO: `/v3/projects/de754769f766487e859de1729e9d7a77` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:10:14.286663 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c912e27-fd46-43ac-bf95-d61057e5d7ac admin admin] RBAC: Authorizing `identity:delete_project(project_id=de754769f766487e859de1729e9d7a77)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:10:14.297136 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c912e27-fd46-43ac-bf95-d61057e5d7ac admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:10:14.455639 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-2c912e27-fd46-43ac-bf95-d61057e5d7ac admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'de754769f766487e859de1729e9d7a77', 'request_id': 'req-2c912e27-fd46-43ac-bf95-d61057e5d7ac'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:10:14.471563 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-2c912e27-fd46-43ac-bf95-d61057e5d7ac admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'de754769f766487e859de1729e9d7a77', 'request_id': 'req-2c912e27-fd46-43ac-bf95-d61057e5d7ac'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:10:14.482756 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-2c912e27-fd46-43ac-bf95-d61057e5d7ac admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'de754769f766487e859de1729e9d7a77', 'request_id': 'req-2c912e27-fd46-43ac-bf95-d61057e5d7ac'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:10:14.519208 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-2c912e27-fd46-43ac-bf95-d61057e5d7ac admin admin] The token cache is being invalidate because project de754769f766487e859de1729e9d7a77 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81424) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 11:10:14.519911 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-2c912e27-fd46-43ac-bf95-d61057e5d7ac admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:10:14.521108 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-2c912e27-fd46-43ac-bf95-d61057e5d7ac admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:10:14.523661 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1240/2462] 10.222.0.40 () {58 vars in 1233 bytes} [Fri Aug 26 11:10:14 2022] DELETE /identity/v3/projects/de754769f766487e859de1729e9d7a77 => generated 0 bytes in 450 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 11:10:14.840800 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1b787a-b361-4fa0-a228-e269a489a750 admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:10:14.844572 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-0e1b787a-b361-4fa0-a228-e269a489a750 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:10:14.956572 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1b787a-b361-4fa0-a228-e269a489a750 admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:10:15.051035 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17012c22-a9f1-47db-8804-171fd4965820 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91da2acd2534423091537e1ba068d2b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e30ccf0c1204727b786876c141dbcd2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:10:15.051706 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17012c22-a9f1-47db-8804-171fd4965820 service neutron] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:10:15.051986 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17012c22-a9f1-47db-8804-171fd4965820 service neutron] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:10:15.052260 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17012c22-a9f1-47db-8804-171fd4965820 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:10:15.054860 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-17012c22-a9f1-47db-8804-171fd4965820 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:10:15.083481 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17012c22-a9f1-47db-8804-171fd4965820 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:10:15.085308 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17012c22-a9f1-47db-8804-171fd4965820 service neutron] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:10:15.098495 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1223/2463] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 11:10:14 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 258 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 11:10:15.612527 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c912e27-fd46-43ac-bf95-d61057e5d7ac admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:10:15.617400 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-2c912e27-fd46-43ac-bf95-d61057e5d7ac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:10:15.884499 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c912e27-fd46-43ac-bf95-d61057e5d7ac admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:10:15.902793 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674d872f-5488-49a9-b5d3-9d4bf78a8202 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f4b4a662db049ecacc865ecff10bdb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e30ccf0c1204727b786876c141dbcd2', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:10:15.903669 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674d872f-5488-49a9-b5d3-9d4bf78a8202 service nova] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:10:15.904048 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674d872f-5488-49a9-b5d3-9d4bf78a8202 service nova] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:10:15.904415 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674d872f-5488-49a9-b5d3-9d4bf78a8202 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:10:15.914865 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-674d872f-5488-49a9-b5d3-9d4bf78a8202 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:10:15.931358 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-674d872f-5488-49a9-b5d3-9d4bf78a8202 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:10:15.933595 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-674d872f-5488-49a9-b5d3-9d4bf78a8202 service nova] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:10:15.957118 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1241/2464] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 11:10:15 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 11:10:16.125130 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2b3ba5-82e4-4159-be78-4b6b5773359a None None] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:10:16.125696 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2b3ba5-82e4-4159-be78-4b6b5773359a None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:10:16.125696 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2b3ba5-82e4-4159-be78-4b6b5773359a None None] PATH_INFO: `/` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:10:16.127406 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1224/2465] 10.222.0.40 () {54 vars in 961 bytes} [Fri Aug 26 11:10:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 11:10:16.144609 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a604635e-6a9d-417e-bb8f-d06cbf687ba7 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:10:16.144871 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a604635e-6a9d-417e-bb8f-d06cbf687ba7 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:10:16.145128 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a604635e-6a9d-417e-bb8f-d06cbf687ba7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:10:16.153914 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 11:10:16.180067 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-a604635e-6a9d-417e-bb8f-d06cbf687ba7 None None] MFA Rules not processed for user `f29c5dfed31f42a9b5cbc0cd72e9fdc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 11:10:16.183449 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-a604635e-6a9d-417e-bb8f-d06cbf687ba7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:10:16.291435 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1242/2466] 10.222.0.40 () {58 vars in 1058 bytes} [Fri Aug 26 11:10:16 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 11:10:16.291847 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2b3ba5-82e4-4159-be78-4b6b5773359a None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:10:16.315645 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2b3ba5-82e4-4159-be78-4b6b5773359a None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:10:16.322035 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0b65e7-4075-44bf-86e1-a53c4cd11ff1 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f29c5dfed31f42a9b5cbc0cd72e9fdc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e30ccf0c1204727b786876c141dbcd2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:10:16.322745 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0b65e7-4075-44bf-86e1-a53c4cd11ff1 service placement] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:10:16.323093 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0b65e7-4075-44bf-86e1-a53c4cd11ff1 service placement] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:10:16.323434 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0b65e7-4075-44bf-86e1-a53c4cd11ff1 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:10:16.325335 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-5f0b65e7-4075-44bf-86e1-a53c4cd11ff1 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:10:16.345191 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f0b65e7-4075-44bf-86e1-a53c4cd11ff1 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:10:16.347800 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f0b65e7-4075-44bf-86e1-a53c4cd11ff1 service placement] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:10:16.369733 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1225/2467] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 11:10:16 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 79 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 26 11:10:18.946096 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a604635e-6a9d-417e-bb8f-d06cbf687ba7 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:10:18.948258 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-a604635e-6a9d-417e-bb8f-d06cbf687ba7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:10:18.961401 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a604635e-6a9d-417e-bb8f-d06cbf687ba7 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:10:18.965177 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6473e915-c4a9-4b45-8e2c-1da0e69530ec service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91da2acd2534423091537e1ba068d2b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e30ccf0c1204727b786876c141dbcd2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:10:18.965802 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6473e915-c4a9-4b45-8e2c-1da0e69530ec service neutron] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:10:18.966114 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6473e915-c4a9-4b45-8e2c-1da0e69530ec service neutron] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:10:18.966582 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6473e915-c4a9-4b45-8e2c-1da0e69530ec service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:10:18.969504 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-6473e915-c4a9-4b45-8e2c-1da0e69530ec service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:10:18.988599 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6473e915-c4a9-4b45-8e2c-1da0e69530ec service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:10:18.990238 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6473e915-c4a9-4b45-8e2c-1da0e69530ec service neutron] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:10:19.004388 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1243/2468] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 11:10:18 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 26 11:10:19.632294 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf439e2c-d927-4378-af1b-4c9538f5e7df None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:10:19.632770 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf439e2c-d927-4378-af1b-4c9538f5e7df None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:10:19.632885 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf439e2c-d927-4378-af1b-4c9538f5e7df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:10:19.655444 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-bf439e2c-d927-4378-af1b-4c9538f5e7df None None] MFA Rules not processed for user `163e8be86d5640f88893c50e6f3a8af7`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 11:10:19.656903 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-bf439e2c-d927-4378-af1b-4c9538f5e7df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:10:19.727533 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1226/2469] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 11:10:19 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 11:10:19.782090 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6473e915-c4a9-4b45-8e2c-1da0e69530ec service neutron] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:10:19.796042 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6473e915-c4a9-4b45-8e2c-1da0e69530ec service neutron] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:10:19.813187 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f83962-74d2-4950-bd46-20f4163f1c99 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f4b4a662db049ecacc865ecff10bdb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e30ccf0c1204727b786876c141dbcd2', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:10:19.814316 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f83962-74d2-4950-bd46-20f4163f1c99 service nova] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:10:19.814677 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f83962-74d2-4950-bd46-20f4163f1c99 service nova] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:10:19.814951 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f83962-74d2-4950-bd46-20f4163f1c99 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:10:19.817174 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-46f83962-74d2-4950-bd46-20f4163f1c99 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:10:19.924060 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f83962-74d2-4950-bd46-20f4163f1c99 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:10:19.928168 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f83962-74d2-4950-bd46-20f4163f1c99 service nova] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:10:19.939688 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1244/2470] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 11:10:19 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 11:10:26.080449 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf439e2c-d927-4378-af1b-4c9538f5e7df None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:10:26.103454 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf439e2c-d927-4378-af1b-4c9538f5e7df None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:10:26.109359 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2fbd53-df39-4a4a-b694-9a66c3bd6e44 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f4b4a662db049ecacc865ecff10bdb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e30ccf0c1204727b786876c141dbcd2', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:10:26.110025 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2fbd53-df39-4a4a-b694-9a66c3bd6e44 service nova] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:10:26.110367 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2fbd53-df39-4a4a-b694-9a66c3bd6e44 service nova] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:10:26.110643 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2fbd53-df39-4a4a-b694-9a66c3bd6e44 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:10:26.112480 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-6d2fbd53-df39-4a4a-b694-9a66c3bd6e44 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:10:26.126455 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d2fbd53-df39-4a4a-b694-9a66c3bd6e44 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:10:26.128394 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d2fbd53-df39-4a4a-b694-9a66c3bd6e44 service nova] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:10:26.140577 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1227/2471] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 11:10:26 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 61 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 11:10:29.864473 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e033615b-3697-4e01-9421-c6075035e310 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:10:29.865050 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e033615b-3697-4e01-9421-c6075035e310 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:10:29.865050 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e033615b-3697-4e01-9421-c6075035e310 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:10:29.890907 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-e033615b-3697-4e01-9421-c6075035e310 None None] MFA Rules not processed for user `9e8355dc324e49d881b9f0fc5c2923ad`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 11:10:29.892192 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-e033615b-3697-4e01-9421-c6075035e310 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:10:29.959793 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1245/2472] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 11:10:29 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 11:10:35.137399 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2fbd53-df39-4a4a-b694-9a66c3bd6e44 service nova] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:10:35.142915 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-6d2fbd53-df39-4a4a-b694-9a66c3bd6e44 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:10:35.166654 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2fbd53-df39-4a4a-b694-9a66c3bd6e44 service nova] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:10:35.176840 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e26172b-0a3e-4bbe-b72c-0160c94aa610 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f4b4a662db049ecacc865ecff10bdb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e30ccf0c1204727b786876c141dbcd2', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:10:35.177825 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e26172b-0a3e-4bbe-b72c-0160c94aa610 service nova] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:10:35.178160 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e26172b-0a3e-4bbe-b72c-0160c94aa610 service nova] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:10:35.178430 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e26172b-0a3e-4bbe-b72c-0160c94aa610 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:10:35.180747 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-9e26172b-0a3e-4bbe-b72c-0160c94aa610 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:10:35.196695 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e26172b-0a3e-4bbe-b72c-0160c94aa610 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:10:35.198642 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e26172b-0a3e-4bbe-b72c-0160c94aa610 service nova] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:10:35.216900 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1228/2473] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 11:10:35 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 11:10:39.951978 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=87 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 11:10:45.207089 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=94,keystone:DELETE=2,keystone:INSERT=1 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 11:10:46.241519 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7ddaa7-fd9d-4ab7-8040-7842adc1a0c1 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:10:46.242068 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7ddaa7-fd9d-4ab7-8040-7842adc1a0c1 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:10:46.242068 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7ddaa7-fd9d-4ab7-8040-7842adc1a0c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:10:46.265966 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-ed7ddaa7-fd9d-4ab7-8040-7842adc1a0c1 None None] MFA Rules not processed for user `50be12ccbd274a8792615e9dd10b61ca`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 11:10:46.267254 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-ed7ddaa7-fd9d-4ab7-8040-7842adc1a0c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:10:46.363686 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1246/2474] 10.222.0.40 () {58 vars in 1009 bytes} [Fri Aug 26 11:10:46 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 11:10:46.456433 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e26172b-0a3e-4bbe-b72c-0160c94aa610 service nova] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:10:46.458818 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-9e26172b-0a3e-4bbe-b72c-0160c94aa610 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:10:46.577802 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e26172b-0a3e-4bbe-b72c-0160c94aa610 service nova] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:10:46.583953 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bad939f-d64c-4a32-bab8-9a629adcbae0 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6529bea296c9482a973ced5fe9f2212f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e30ccf0c1204727b786876c141dbcd2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:10:46.584940 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bad939f-d64c-4a32-bab8-9a629adcbae0 service swift] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:10:46.585408 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bad939f-d64c-4a32-bab8-9a629adcbae0 service swift] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:10:46.585909 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bad939f-d64c-4a32-bab8-9a629adcbae0 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:10:46.599402 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bad939f-d64c-4a32-bab8-9a629adcbae0 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:10:46.601807 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bad939f-d64c-4a32-bab8-9a629adcbae0 service swift] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:10:46.613054 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1229/2475] 10.222.0.40 () {60 vars in 1384 bytes} [Fri Aug 26 11:10:46 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 26 11:10:48.726037 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7ddaa7-fd9d-4ab7-8040-7842adc1a0c1 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:10:48.727807 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-ed7ddaa7-fd9d-4ab7-8040-7842adc1a0c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:10:48.743182 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7ddaa7-fd9d-4ab7-8040-7842adc1a0c1 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:10:48.750687 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f162fd0a-1c67-44e5-878d-4edf96117f8b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91da2acd2534423091537e1ba068d2b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e30ccf0c1204727b786876c141dbcd2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:10:48.751656 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f162fd0a-1c67-44e5-878d-4edf96117f8b service neutron] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:10:48.751951 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f162fd0a-1c67-44e5-878d-4edf96117f8b service neutron] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:10:48.752229 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f162fd0a-1c67-44e5-878d-4edf96117f8b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:10:48.754824 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-f162fd0a-1c67-44e5-878d-4edf96117f8b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:10:48.892053 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f162fd0a-1c67-44e5-878d-4edf96117f8b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:10:48.893689 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f162fd0a-1c67-44e5-878d-4edf96117f8b service neutron] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:10:48.909509 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1247/2476] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 11:10:48 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 184 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 11:10:56.609179 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=32 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 11:10:58.900449 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=55 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 11:11:01.320626 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bad939f-d64c-4a32-bab8-9a629adcbae0 service swift] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:11:01.330209 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bad939f-d64c-4a32-bab8-9a629adcbae0 service swift] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:11:01.335695 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa21c3f-38ce-4554-8555-f83070c43d24 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91da2acd2534423091537e1ba068d2b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e30ccf0c1204727b786876c141dbcd2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:11:01.336406 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa21c3f-38ce-4554-8555-f83070c43d24 service neutron] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:11:01.336690 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa21c3f-38ce-4554-8555-f83070c43d24 service neutron] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:11:01.337083 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa21c3f-38ce-4554-8555-f83070c43d24 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:11:01.339088 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-baa21c3f-38ce-4554-8555-f83070c43d24 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:11:01.350105 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa21c3f-38ce-4554-8555-f83070c43d24 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:11:01.351581 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa21c3f-38ce-4554-8555-f83070c43d24 service neutron] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:11:01.361439 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1230/2477] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 11:11:01 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 11:11:01.513892 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b82c7c-b2b6-4fe1-b628-4afc36d4955b None None] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:11:01.515400 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b82c7c-b2b6-4fe1-b628-4afc36d4955b None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:11:01.515400 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b82c7c-b2b6-4fe1-b628-4afc36d4955b None None] PATH_INFO: `/` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:11:01.518206 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1248/2478] 10.222.0.40 () {54 vars in 921 bytes} [Fri Aug 26 11:11:01 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 11:11:01.526640 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6673fa-9c5f-472f-a777-3cabaad69e38 None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:11:01.527092 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6673fa-9c5f-472f-a777-3cabaad69e38 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:11:01.527603 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6673fa-9c5f-472f-a777-3cabaad69e38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:11:01.552254 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-9b6673fa-9c5f-472f-a777-3cabaad69e38 None None] MFA Rules not processed for user `1f4b4a662db049ecacc865ecff10bdb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 11:11:01.553529 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-9b6673fa-9c5f-472f-a777-3cabaad69e38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:11:01.564466 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1231/2479] 10.222.0.40 () {58 vars in 1018 bytes} [Fri Aug 26 11:11:01 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 11:11:01.582650 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b82c7c-b2b6-4fe1-b628-4afc36d4955b None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:11:01.584387 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-94b82c7c-b2b6-4fe1-b628-4afc36d4955b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:11:01.656673 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b82c7c-b2b6-4fe1-b628-4afc36d4955b None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:11:01.660618 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7777f01d-17bb-40fb-bd31-687448a137c8 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8631d7ecfd2849daa8694aecccd22a05', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e30ccf0c1204727b786876c141dbcd2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:11:01.661309 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7777f01d-17bb-40fb-bd31-687448a137c8 service glance] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:11:01.661602 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7777f01d-17bb-40fb-bd31-687448a137c8 service glance] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:11:01.661879 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7777f01d-17bb-40fb-bd31-687448a137c8 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:11:01.673550 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7777f01d-17bb-40fb-bd31-687448a137c8 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:11:01.675010 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7777f01d-17bb-40fb-bd31-687448a137c8 service glance] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:11:01.687811 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1249/2480] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 11:11:01 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 11:11:01.762921 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c47c688-16a5-4fb9-88ff-d514a5bf07a9 None None] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:11:01.763233 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c47c688-16a5-4fb9-88ff-d514a5bf07a9 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:11:01.763539 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c47c688-16a5-4fb9-88ff-d514a5bf07a9 None None] PATH_INFO: `/` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:11:01.765575 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1232/2481] 10.222.0.40 () {54 vars in 921 bytes} [Fri Aug 26 11:11:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 11:11:01.770546 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb04d83-1f7a-420b-aac2-79c8a5ecb8d9 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:11:01.770832 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb04d83-1f7a-420b-aac2-79c8a5ecb8d9 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:11:01.771109 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb04d83-1f7a-420b-aac2-79c8a5ecb8d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:11:01.791661 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-8bb04d83-1f7a-420b-aac2-79c8a5ecb8d9 None None] MFA Rules not processed for user `91da2acd2534423091537e1ba068d2b7`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 11:11:01.792865 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-8bb04d83-1f7a-420b-aac2-79c8a5ecb8d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:11:01.799608 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1250/2482] 10.222.0.40 () {58 vars in 1018 bytes} [Fri Aug 26 11:11:01 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 11:11:01.811704 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c47c688-16a5-4fb9-88ff-d514a5bf07a9 None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:11:01.822959 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c47c688-16a5-4fb9-88ff-d514a5bf07a9 None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:11:01.827102 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94aa60e-7699-4f77-8bff-ee844c67643a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91da2acd2534423091537e1ba068d2b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e30ccf0c1204727b786876c141dbcd2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:11:01.828023 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94aa60e-7699-4f77-8bff-ee844c67643a service neutron] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:11:01.828522 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94aa60e-7699-4f77-8bff-ee844c67643a service neutron] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:11:01.828772 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94aa60e-7699-4f77-8bff-ee844c67643a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:11:01.836515 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b94aa60e-7699-4f77-8bff-ee844c67643a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:11:01.838058 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b94aa60e-7699-4f77-8bff-ee844c67643a service neutron] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:11:01.846570 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1233/2483] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 11:11:01 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 11:11:01.941888 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e65b15-ef46-45a9-8027-048ba282e978 None None] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:11:01.942458 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e65b15-ef46-45a9-8027-048ba282e978 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:11:01.942942 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e65b15-ef46-45a9-8027-048ba282e978 None None] PATH_INFO: `/` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:11:01.945266 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1251/2484] 10.222.0.40 () {54 vars in 921 bytes} [Fri Aug 26 11:11:01 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 11:11:01.951995 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d911af54-1273-4f60-b445-83f1993cfc1e None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:11:01.952279 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d911af54-1273-4f60-b445-83f1993cfc1e None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:11:01.952540 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d911af54-1273-4f60-b445-83f1993cfc1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:11:01.973744 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-d911af54-1273-4f60-b445-83f1993cfc1e None None] MFA Rules not processed for user `1f4b4a662db049ecacc865ecff10bdb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 11:11:01.974996 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-d911af54-1273-4f60-b445-83f1993cfc1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:11:01.986391 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1234/2485] 10.222.0.40 () {58 vars in 1018 bytes} [Fri Aug 26 11:11:01 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 11:11:01.997507 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e65b15-ef46-45a9-8027-048ba282e978 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:11:02.015097 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e65b15-ef46-45a9-8027-048ba282e978 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:11:02.020750 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d66e5e-c678-4e81-98be-ed78265bb4f6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91da2acd2534423091537e1ba068d2b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e30ccf0c1204727b786876c141dbcd2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:11:02.021688 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d66e5e-c678-4e81-98be-ed78265bb4f6 service neutron] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:11:02.022157 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d66e5e-c678-4e81-98be-ed78265bb4f6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:11:02.022621 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d66e5e-c678-4e81-98be-ed78265bb4f6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:11:02.035110 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d66e5e-c678-4e81-98be-ed78265bb4f6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:11:02.036589 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d66e5e-c678-4e81-98be-ed78265bb4f6 service neutron] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:11:02.050756 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1252/2486] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 11:11:01 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 11:11:04.120476 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16081f06-d1e4-4d80-b40e-c16e86971418 None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:11:04.120943 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16081f06-d1e4-4d80-b40e-c16e86971418 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:11:04.121055 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16081f06-d1e4-4d80-b40e-c16e86971418 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:11:04.142115 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-16081f06-d1e4-4d80-b40e-c16e86971418 None None] MFA Rules not processed for user `50be12ccbd274a8792615e9dd10b61ca`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 11:11:04.143271 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-16081f06-d1e4-4d80-b40e-c16e86971418 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:11:04.150793 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1235/2487] 10.222.0.40 () {58 vars in 1009 bytes} [Fri Aug 26 11:11:04 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 11:11:04.163852 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d66e5e-c678-4e81-98be-ed78265bb4f6 service neutron] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:11:04.175413 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d66e5e-c678-4e81-98be-ed78265bb4f6 service neutron] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:11:04.180275 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b86b52-60c6-40c0-bcdd-a2ae364797fb service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6529bea296c9482a973ced5fe9f2212f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e30ccf0c1204727b786876c141dbcd2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:11:04.180866 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b86b52-60c6-40c0-bcdd-a2ae364797fb service swift] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:11:04.181139 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b86b52-60c6-40c0-bcdd-a2ae364797fb service swift] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:11:04.181408 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b86b52-60c6-40c0-bcdd-a2ae364797fb service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:11:04.189413 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b86b52-60c6-40c0-bcdd-a2ae364797fb service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:11:04.191132 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b86b52-60c6-40c0-bcdd-a2ae364797fb service swift] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:11:04.200007 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1253/2488] 10.222.0.40 () {60 vars in 1384 bytes} [Fri Aug 26 11:11:04 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 26 11:11:14.135984 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=21 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 11:11:14.196950 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=41 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 11:11:19.956758 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ddea904-2d3c-477a-80ad-271b14fdecff None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:11:19.957282 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ddea904-2d3c-477a-80ad-271b14fdecff None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:11:19.957471 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ddea904-2d3c-477a-80ad-271b14fdecff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:11:19.986372 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-2ddea904-2d3c-477a-80ad-271b14fdecff None None] MFA Rules not processed for user `50be12ccbd274a8792615e9dd10b61ca`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 11:11:19.987616 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-2ddea904-2d3c-477a-80ad-271b14fdecff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:11:19.995360 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1236/2489] 10.222.0.40 () {58 vars in 967 bytes} [Fri Aug 26 11:11:19 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 11:11:20.010413 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b86b52-60c6-40c0-bcdd-a2ae364797fb service swift] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:11:20.021844 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b86b52-60c6-40c0-bcdd-a2ae364797fb service swift] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:11:20.027234 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49569e6-7454-4b67-a22f-92bc7206c526 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6529bea296c9482a973ced5fe9f2212f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e30ccf0c1204727b786876c141dbcd2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:11:20.027859 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49569e6-7454-4b67-a22f-92bc7206c526 service swift] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:11:20.028138 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49569e6-7454-4b67-a22f-92bc7206c526 service swift] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:11:20.028410 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49569e6-7454-4b67-a22f-92bc7206c526 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:11:20.038982 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a49569e6-7454-4b67-a22f-92bc7206c526 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:11:20.040871 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a49569e6-7454-4b67-a22f-92bc7206c526 service swift] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:11:20.051178 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1254/2490] 10.222.0.40 () {60 vars in 1384 bytes} [Fri Aug 26 11:11:20 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 26 11:11:24.858028 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62202ddb-97b2-4b5a-8b46-fac36a1e0bc2 None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:11:24.858851 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62202ddb-97b2-4b5a-8b46-fac36a1e0bc2 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:11:24.859059 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62202ddb-97b2-4b5a-8b46-fac36a1e0bc2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:11:24.879406 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-62202ddb-97b2-4b5a-8b46-fac36a1e0bc2 None None] MFA Rules not processed for user `85716f53e88048af9a8fdcbe4415979c`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 11:11:24.880634 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-62202ddb-97b2-4b5a-8b46-fac36a1e0bc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:11:24.937269 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1237/2491] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 11:11:24 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 11:11:30.047198 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=6 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 11:11:34.930328 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=28 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 11:11:51.984572 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49569e6-7454-4b67-a22f-92bc7206c526 service swift] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:11:51.994160 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49569e6-7454-4b67-a22f-92bc7206c526 service swift] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:11:51.998632 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce803ecf-469c-46c7-a93d-56204af08f2f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91da2acd2534423091537e1ba068d2b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e30ccf0c1204727b786876c141dbcd2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:11:51.999231 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce803ecf-469c-46c7-a93d-56204af08f2f service neutron] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:11:51.999510 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce803ecf-469c-46c7-a93d-56204af08f2f service neutron] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:11:51.999785 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce803ecf-469c-46c7-a93d-56204af08f2f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:11:52.001540 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-ce803ecf-469c-46c7-a93d-56204af08f2f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:11:52.069786 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce803ecf-469c-46c7-a93d-56204af08f2f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:11:52.071428 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce803ecf-469c-46c7-a93d-56204af08f2f service neutron] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:11:52.083587 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1255/2492] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 11:11:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 11:11:55.124041 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62202ddb-97b2-4b5a-8b46-fac36a1e0bc2 None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:11:55.125854 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-62202ddb-97b2-4b5a-8b46-fac36a1e0bc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:11:55.153546 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62202ddb-97b2-4b5a-8b46-fac36a1e0bc2 None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:11:55.160817 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ba7234-13bb-4162-a84b-7bda4913d414 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f29c5dfed31f42a9b5cbc0cd72e9fdc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e30ccf0c1204727b786876c141dbcd2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:11:55.161439 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ba7234-13bb-4162-a84b-7bda4913d414 service placement] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:11:55.161710 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ba7234-13bb-4162-a84b-7bda4913d414 service placement] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:11:55.162066 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ba7234-13bb-4162-a84b-7bda4913d414 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:11:55.163791 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-83ba7234-13bb-4162-a84b-7bda4913d414 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:11:55.178211 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ba7234-13bb-4162-a84b-7bda4913d414 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:11:55.179927 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ba7234-13bb-4162-a84b-7bda4913d414 service placement] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:11:55.188219 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce803ecf-469c-46c7-a93d-56204af08f2f service neutron] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:11:55.190849 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-ce803ecf-469c-46c7-a93d-56204af08f2f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:11:55.202083 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1238/2493] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 11:11:55 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 77 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 11:11:55.207714 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce803ecf-469c-46c7-a93d-56204af08f2f service neutron] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:11:55.213142 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3270dbc-6edb-484f-8272-32cea19237a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:11:55.213815 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3270dbc-6edb-484f-8272-32cea19237a9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:11:55.214121 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3270dbc-6edb-484f-8272-32cea19237a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:11:55.214561 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3270dbc-6edb-484f-8272-32cea19237a9 admin admin] PATH_INFO: `/v3/users/163e8be86d5640f88893c50e6f3a8af7` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:11:55.215889 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3270dbc-6edb-484f-8272-32cea19237a9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=163e8be86d5640f88893c50e6f3a8af7)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:11:55.219756 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3270dbc-6edb-484f-8272-32cea19237a9 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:11:55.274059 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-c3270dbc-6edb-484f-8272-32cea19237a9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '163e8be86d5640f88893c50e6f3a8af7'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:11:55.274630 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-c3270dbc-6edb-484f-8272-32cea19237a9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '163e8be86d5640f88893c50e6f3a8af7'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:11:55.282652 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-c3270dbc-6edb-484f-8272-32cea19237a9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '163e8be86d5640f88893c50e6f3a8af7'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:11:55.283396 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-c3270dbc-6edb-484f-8272-32cea19237a9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '163e8be86d5640f88893c50e6f3a8af7'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:11:55.296438 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-c3270dbc-6edb-484f-8272-32cea19237a9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '163e8be86d5640f88893c50e6f3a8af7'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:11:55.311228 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1256/2494] 10.222.0.40 () {58 vars in 1224 bytes} [Fri Aug 26 11:11:55 2022] DELETE /identity/v3/users/163e8be86d5640f88893c50e6f3a8af7 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 26 11:11:55.491592 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ba7234-13bb-4162-a84b-7bda4913d414 service placement] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:11:55.504841 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ba7234-13bb-4162-a84b-7bda4913d414 service placement] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:11:55.565214 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba8f593-4011-4531-b38a-e4301fe6c3ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:11:55.565842 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba8f593-4011-4531-b38a-e4301fe6c3ac admin admin] REQUEST_METHOD: `DELETE` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:11:55.566152 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba8f593-4011-4531-b38a-e4301fe6c3ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:11:55.566415 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba8f593-4011-4531-b38a-e4301fe6c3ac admin admin] PATH_INFO: `/v3/projects/0fa496929354424ca7d024a8279f2e80` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:11:55.567539 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba8f593-4011-4531-b38a-e4301fe6c3ac admin admin] RBAC: Authorizing `identity:delete_project(project_id=0fa496929354424ca7d024a8279f2e80)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:11:55.569416 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba8f593-4011-4531-b38a-e4301fe6c3ac admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:11:55.618873 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-fba8f593-4011-4531-b38a-e4301fe6c3ac admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0fa496929354424ca7d024a8279f2e80', 'request_id': 'req-fba8f593-4011-4531-b38a-e4301fe6c3ac'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:11:55.626456 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-fba8f593-4011-4531-b38a-e4301fe6c3ac admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0fa496929354424ca7d024a8279f2e80', 'request_id': 'req-fba8f593-4011-4531-b38a-e4301fe6c3ac'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:11:55.633102 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-fba8f593-4011-4531-b38a-e4301fe6c3ac admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0fa496929354424ca7d024a8279f2e80', 'request_id': 'req-fba8f593-4011-4531-b38a-e4301fe6c3ac'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:11:55.646767 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-fba8f593-4011-4531-b38a-e4301fe6c3ac admin admin] The token cache is being invalidate because project 0fa496929354424ca7d024a8279f2e80 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81423) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 11:11:55.646767 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-fba8f593-4011-4531-b38a-e4301fe6c3ac admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:11:55.646767 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-fba8f593-4011-4531-b38a-e4301fe6c3ac admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:11:55.648169 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1239/2495] 10.222.0.40 () {58 vars in 1233 bytes} [Fri Aug 26 11:11:55 2022] DELETE /identity/v3/projects/0fa496929354424ca7d024a8279f2e80 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 11:11:55.653038 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1072acf3-9934-4bb7-ad25-4768c66e8ac2 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:11:55.653339 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1072acf3-9934-4bb7-ad25-4768c66e8ac2 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:11:55.653666 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1072acf3-9934-4bb7-ad25-4768c66e8ac2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:11:55.673617 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-1072acf3-9934-4bb7-ad25-4768c66e8ac2 None None] MFA Rules not processed for user `ca155a48e2024c32af3fd2e19e5fac11`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 11:11:55.674877 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-1072acf3-9934-4bb7-ad25-4768c66e8ac2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:11:55.736993 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1257/2496] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 11:11:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 11:11:55.745225 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a380d5-21db-48bc-b87a-d738a2740993 None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:11:55.745511 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a380d5-21db-48bc-b87a-d738a2740993 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:11:55.745786 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a380d5-21db-48bc-b87a-d738a2740993 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:11:55.767338 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-16a380d5-21db-48bc-b87a-d738a2740993 None None] MFA Rules not processed for user `ca155a48e2024c32af3fd2e19e5fac11`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 11:11:55.768600 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-16a380d5-21db-48bc-b87a-d738a2740993 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:11:55.776872 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1240/2497] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 11:11:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 11:11:55.781391 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1072acf3-9934-4bb7-ad25-4768c66e8ac2 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:11:55.792183 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1072acf3-9934-4bb7-ad25-4768c66e8ac2 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:11:55.797181 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a3a001-32c5-4985-8927-5577fd6b76cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:11:55.797794 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a3a001-32c5-4985-8927-5577fd6b76cb admin admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:11:55.798110 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a3a001-32c5-4985-8927-5577fd6b76cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:11:55.798414 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a3a001-32c5-4985-8927-5577fd6b76cb admin admin] PATH_INFO: `/v3/domains` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:11:55.799097 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a3a001-32c5-4985-8927-5577fd6b76cb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 11:11:55.799397 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a3a001-32c5-4985-8927-5577fd6b76cb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:11:55.801000 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a3a001-32c5-4985-8927-5577fd6b76cb admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:11:55.802699 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 11:11:55.818036 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1258/2498] 10.222.0.40 () {58 vars in 1153 bytes} [Fri Aug 26 11:11:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 11:11:55.819855 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a380d5-21db-48bc-b87a-d738a2740993 None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:11:55.831754 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a380d5-21db-48bc-b87a-d738a2740993 None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:11:55.837769 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4374cfc2-5294-4f62-a01b-2d55aaef1b33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:11:55.838423 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4374cfc2-5294-4f62-a01b-2d55aaef1b33 admin admin] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:11:55.838705 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4374cfc2-5294-4f62-a01b-2d55aaef1b33 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:11:55.838975 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4374cfc2-5294-4f62-a01b-2d55aaef1b33 admin admin] PATH_INFO: `/v3/projects` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:11:55.839609 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4374cfc2-5294-4f62-a01b-2d55aaef1b33 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:11:55.841558 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4374cfc2-5294-4f62-a01b-2d55aaef1b33 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:11:55.875385 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1241/2499] 10.222.0.40 () {60 vars in 1153 bytes} [Fri Aug 26 11:11:55 2022] POST /identity/v3/projects => generated 363 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 11:11:55.884493 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a3a001-32c5-4985-8927-5577fd6b76cb admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:11:55.896903 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a3a001-32c5-4985-8927-5577fd6b76cb admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:11:55.957663 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69ae1cf-0cdf-4d77-9806-8a55ac774623 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:11:55.958306 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69ae1cf-0cdf-4d77-9806-8a55ac774623 admin admin] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:11:55.958580 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69ae1cf-0cdf-4d77-9806-8a55ac774623 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:11:55.958848 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69ae1cf-0cdf-4d77-9806-8a55ac774623 admin admin] PATH_INFO: `/v3/users` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:11:55.959481 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e69ae1cf-0cdf-4d77-9806-8a55ac774623 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:11:55.961428 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e69ae1cf-0cdf-4d77-9806-8a55ac774623 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:11:55.993793 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1259/2500] 10.222.0.40 () {60 vars in 1144 bytes} [Fri Aug 26 11:11:55 2022] POST /identity/v3/users => generated 282 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 26 11:11:55.995111 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4374cfc2-5294-4f62-a01b-2d55aaef1b33 admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:11:56.015586 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4374cfc2-5294-4f62-a01b-2d55aaef1b33 admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:11:56.020553 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93da20e-79a5-4cbb-9c39-153794a2785e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:11:56.021173 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93da20e-79a5-4cbb-9c39-153794a2785e admin admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:11:56.021458 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93da20e-79a5-4cbb-9c39-153794a2785e admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:11:56.021729 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93da20e-79a5-4cbb-9c39-153794a2785e admin admin] PATH_INFO: `/v3/roles` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:11:56.022739 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f93da20e-79a5-4cbb-9c39-153794a2785e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:11:56.024508 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f93da20e-79a5-4cbb-9c39-153794a2785e admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:11:56.040103 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1242/2501] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 26 11:11:55 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 26 11:11:56.044757 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69ae1cf-0cdf-4d77-9806-8a55ac774623 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:11:56.055650 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69ae1cf-0cdf-4d77-9806-8a55ac774623 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:11:56.060361 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb76e08-16a1-4508-a6b5-cf436f1f7659 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:11:56.060981 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb76e08-16a1-4508-a6b5-cf436f1f7659 admin admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:11:56.061257 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb76e08-16a1-4508-a6b5-cf436f1f7659 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:11:56.061527 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb76e08-16a1-4508-a6b5-cf436f1f7659 admin admin] PATH_INFO: `/v3/roles` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:11:56.062395 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb76e08-16a1-4508-a6b5-cf436f1f7659 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:11:56.064038 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb76e08-16a1-4508-a6b5-cf436f1f7659 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:11:56.075246 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1260/2502] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 26 11:11:56 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 11:11:56.080546 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93da20e-79a5-4cbb-9c39-153794a2785e admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:11:56.090961 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93da20e-79a5-4cbb-9c39-153794a2785e admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:11:56.096910 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e3a151-2e29-4dc7-9787-5491909c302e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:11:56.097580 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e3a151-2e29-4dc7-9787-5491909c302e admin admin] REQUEST_METHOD: `PUT` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:11:56.097853 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e3a151-2e29-4dc7-9787-5491909c302e admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:11:56.098156 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e3a151-2e29-4dc7-9787-5491909c302e admin admin] PATH_INFO: `/v3/projects/1e54372406304894bb99817f57aab09d/users/f02ff01acf35478ba73c99a1adbcc005/roles/23e174ce743b46eabe618c212691da55` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:11:56.117307 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e3a151-2e29-4dc7-9787-5491909c302e admin admin] RBAC: Authorizing `identity:create_grant(project_id=1e54372406304894bb99817f57aab09d, user_id=f02ff01acf35478ba73c99a1adbcc005, role_id=23e174ce743b46eabe618c212691da55)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:11:56.120129 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e3a151-2e29-4dc7-9787-5491909c302e admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:11:56.138483 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1243/2503] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 26 11:11:56 2022] PUT /identity/v3/projects/1e54372406304894bb99817f57aab09d/users/f02ff01acf35478ba73c99a1adbcc005/roles/23e174ce743b46eabe618c212691da55 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 11:11:56.150401 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946f19b3-dc30-490d-9d46-43905754bb1c None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:11:56.150686 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946f19b3-dc30-490d-9d46-43905754bb1c None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:11:56.150958 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946f19b3-dc30-490d-9d46-43905754bb1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:11:56.179825 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-946f19b3-dc30-490d-9d46-43905754bb1c None None] MFA Rules not processed for user `f02ff01acf35478ba73c99a1adbcc005`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 11:11:56.181406 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-946f19b3-dc30-490d-9d46-43905754bb1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:11:56.248192 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1261/2504] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 11:11:56 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 11:11:56.252852 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e3a151-2e29-4dc7-9787-5491909c302e admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:11:56.267352 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 11:11:56.323332 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e3a151-2e29-4dc7-9787-5491909c302e admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:11:56.329861 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721d6cd7-53f2-4814-be45-bdef626bc753 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:11:56.330512 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721d6cd7-53f2-4814-be45-bdef626bc753 admin admin] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:11:56.330785 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721d6cd7-53f2-4814-be45-bdef626bc753 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:11:56.331053 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721d6cd7-53f2-4814-be45-bdef626bc753 admin admin] PATH_INFO: `/v3/projects` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:11:56.331683 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-721d6cd7-53f2-4814-be45-bdef626bc753 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:11:56.333588 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-721d6cd7-53f2-4814-be45-bdef626bc753 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:11:56.365119 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1244/2505] 10.222.0.40 () {60 vars in 1153 bytes} [Fri Aug 26 11:11:56 2022] POST /identity/v3/projects => generated 365 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 11:11:56.369305 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946f19b3-dc30-490d-9d46-43905754bb1c None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:11:56.381166 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946f19b3-dc30-490d-9d46-43905754bb1c None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:11:56.446866 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1592c761-6979-42cb-a719-4554ac61af7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:11:56.447489 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1592c761-6979-42cb-a719-4554ac61af7c admin admin] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:11:56.447770 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1592c761-6979-42cb-a719-4554ac61af7c admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:11:56.448042 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1592c761-6979-42cb-a719-4554ac61af7c admin admin] PATH_INFO: `/v3/users` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:11:56.448674 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1592c761-6979-42cb-a719-4554ac61af7c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:11:56.450862 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1592c761-6979-42cb-a719-4554ac61af7c admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:11:56.460261 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 11:11:56.474538 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1262/2506] 10.222.0.40 () {60 vars in 1144 bytes} [Fri Aug 26 11:11:56 2022] POST /identity/v3/users => generated 289 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 11:11:56.478826 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721d6cd7-53f2-4814-be45-bdef626bc753 admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:11:56.491147 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721d6cd7-53f2-4814-be45-bdef626bc753 admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:11:56.495944 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc10b85-e132-4226-a9d4-c46893175ebc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:11:56.496554 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc10b85-e132-4226-a9d4-c46893175ebc admin admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:11:56.496827 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc10b85-e132-4226-a9d4-c46893175ebc admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:11:56.498543 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc10b85-e132-4226-a9d4-c46893175ebc admin admin] PATH_INFO: `/v3/roles` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:11:56.498543 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc10b85-e132-4226-a9d4-c46893175ebc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:11:56.499746 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc10b85-e132-4226-a9d4-c46893175ebc admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:11:56.510803 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1245/2507] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 26 11:11:56 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 11:11:56.515669 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1592c761-6979-42cb-a719-4554ac61af7c admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:11:56.526061 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1592c761-6979-42cb-a719-4554ac61af7c admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:11:56.531698 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc4504c-885a-46b4-ad50-b51de3498ef1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:11:56.532364 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc4504c-885a-46b4-ad50-b51de3498ef1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:11:56.532643 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc4504c-885a-46b4-ad50-b51de3498ef1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:11:56.532915 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc4504c-885a-46b4-ad50-b51de3498ef1 admin admin] PATH_INFO: `/v3/projects/121d89e3e8594749a4398243ea1ae3ff/users/aa61400617e44a419668ed25a9b0dd8f/roles/1f97e4602e2e4d6498913a81f9ae8d6a` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:11:56.549576 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc4504c-885a-46b4-ad50-b51de3498ef1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=121d89e3e8594749a4398243ea1ae3ff, user_id=aa61400617e44a419668ed25a9b0dd8f, role_id=1f97e4602e2e4d6498913a81f9ae8d6a)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:11:56.552302 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc4504c-885a-46b4-ad50-b51de3498ef1 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:11:56.565800 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1263/2508] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 26 11:11:56 2022] PUT /identity/v3/projects/121d89e3e8594749a4398243ea1ae3ff/users/aa61400617e44a419668ed25a9b0dd8f/roles/1f97e4602e2e4d6498913a81f9ae8d6a => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 11:11:56.570000 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc10b85-e132-4226-a9d4-c46893175ebc admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:11:56.580918 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc10b85-e132-4226-a9d4-c46893175ebc admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:11:56.639429 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd41a9a-e8ca-49a9-86aa-37b7c04ed35b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:11:56.640163 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd41a9a-e8ca-49a9-86aa-37b7c04ed35b admin admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:11:56.640486 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd41a9a-e8ca-49a9-86aa-37b7c04ed35b admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:11:56.640746 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd41a9a-e8ca-49a9-86aa-37b7c04ed35b admin admin] PATH_INFO: `/v3/roles` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:11:56.641689 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd41a9a-e8ca-49a9-86aa-37b7c04ed35b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:11:56.643314 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd41a9a-e8ca-49a9-86aa-37b7c04ed35b admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:11:56.654384 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1246/2509] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 26 11:11:56 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 11:11:56.658724 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc4504c-885a-46b4-ad50-b51de3498ef1 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:11:56.670046 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc4504c-885a-46b4-ad50-b51de3498ef1 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:11:56.674846 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507f35fa-618c-40f5-a359-0dd3bc901eb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:11:56.675511 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507f35fa-618c-40f5-a359-0dd3bc901eb4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:11:56.675784 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507f35fa-618c-40f5-a359-0dd3bc901eb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:11:56.676053 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507f35fa-618c-40f5-a359-0dd3bc901eb4 admin admin] PATH_INFO: `/v3/projects/121d89e3e8594749a4398243ea1ae3ff/users/aa61400617e44a419668ed25a9b0dd8f/roles/1f97e4602e2e4d6498913a81f9ae8d6a` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:11:56.677952 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-507f35fa-618c-40f5-a359-0dd3bc901eb4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=121d89e3e8594749a4398243ea1ae3ff, user_id=aa61400617e44a419668ed25a9b0dd8f, role_id=1f97e4602e2e4d6498913a81f9ae8d6a)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:11:56.680643 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-507f35fa-618c-40f5-a359-0dd3bc901eb4 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:11:56.691776 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1264/2510] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 26 11:11:56 2022] PUT /identity/v3/projects/121d89e3e8594749a4398243ea1ae3ff/users/aa61400617e44a419668ed25a9b0dd8f/roles/1f97e4602e2e4d6498913a81f9ae8d6a => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 11:11:56.703053 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82174ae4-3fd3-4fa1-8996-a467492a3402 None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:11:56.703334 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82174ae4-3fd3-4fa1-8996-a467492a3402 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:11:56.703607 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82174ae4-3fd3-4fa1-8996-a467492a3402 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:11:56.733040 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-82174ae4-3fd3-4fa1-8996-a467492a3402 None None] MFA Rules not processed for user `aa61400617e44a419668ed25a9b0dd8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 11:11:56.734304 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-82174ae4-3fd3-4fa1-8996-a467492a3402 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:11:56.806885 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1247/2511] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 11:11:56 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 11:11:56.819442 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507f35fa-618c-40f5-a359-0dd3bc901eb4 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:11:56.821223 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-507f35fa-618c-40f5-a359-0dd3bc901eb4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:11:56.885759 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507f35fa-618c-40f5-a359-0dd3bc901eb4 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:11:56.890171 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3aa3bc-8203-4168-b6e8-480ceb3bd05f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c277338a7f814cc6b01c5419775031bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e30ccf0c1204727b786876c141dbcd2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:11:56.892718 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3aa3bc-8203-4168-b6e8-480ceb3bd05f service cinder] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:11:56.892718 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3aa3bc-8203-4168-b6e8-480ceb3bd05f service cinder] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:11:56.892718 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3aa3bc-8203-4168-b6e8-480ceb3bd05f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:11:56.901097 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f3aa3bc-8203-4168-b6e8-480ceb3bd05f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:11:56.902577 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f3aa3bc-8203-4168-b6e8-480ceb3bd05f service cinder] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:11:56.913056 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1265/2512] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 11:11:56 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 11:11:56.994258 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82174ae4-3fd3-4fa1-8996-a467492a3402 None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:11:57.004319 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82174ae4-3fd3-4fa1-8996-a467492a3402 None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:11:57.008062 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7f3970-4c12-46bc-88da-272c6834e8b4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c277338a7f814cc6b01c5419775031bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e30ccf0c1204727b786876c141dbcd2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:11:57.008669 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7f3970-4c12-46bc-88da-272c6834e8b4 service cinder] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:11:57.008942 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7f3970-4c12-46bc-88da-272c6834e8b4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:11:57.009207 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7f3970-4c12-46bc-88da-272c6834e8b4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:11:57.015117 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 11:11:57.070618 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb7f3970-4c12-46bc-88da-272c6834e8b4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:11:57.072044 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb7f3970-4c12-46bc-88da-272c6834e8b4 service cinder] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:11:57.082059 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1248/2513] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 11:11:56 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 11:12:06.605803 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3aa3bc-8203-4168-b6e8-480ceb3bd05f service cinder] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:12:06.670825 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3aa3bc-8203-4168-b6e8-480ceb3bd05f service cinder] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:12:06.676545 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49cfd5c0-95ae-4068-895d-e0eaf8097550 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:12:06.677188 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49cfd5c0-95ae-4068-895d-e0eaf8097550 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:12:06.677466 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49cfd5c0-95ae-4068-895d-e0eaf8097550 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:12:06.677737 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49cfd5c0-95ae-4068-895d-e0eaf8097550 admin admin] PATH_INFO: `/v3/users/f02ff01acf35478ba73c99a1adbcc005` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:12:06.678931 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49cfd5c0-95ae-4068-895d-e0eaf8097550 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f02ff01acf35478ba73c99a1adbcc005)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:12:06.681029 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49cfd5c0-95ae-4068-895d-e0eaf8097550 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:12:06.735015 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-49cfd5c0-95ae-4068-895d-e0eaf8097550 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f02ff01acf35478ba73c99a1adbcc005'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:12:06.735709 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-49cfd5c0-95ae-4068-895d-e0eaf8097550 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f02ff01acf35478ba73c99a1adbcc005'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:12:06.743825 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-49cfd5c0-95ae-4068-895d-e0eaf8097550 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f02ff01acf35478ba73c99a1adbcc005'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:12:06.744533 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-49cfd5c0-95ae-4068-895d-e0eaf8097550 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f02ff01acf35478ba73c99a1adbcc005'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:12:06.746487 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:INSERT=4,keystone:SELECT=89,keystone:DELETE=7 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 11:12:06.760234 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-49cfd5c0-95ae-4068-895d-e0eaf8097550 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f02ff01acf35478ba73c99a1adbcc005'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:12:06.774036 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1266/2514] 10.222.0.40 () {58 vars in 1224 bytes} [Fri Aug 26 11:12:06 2022] DELETE /identity/v3/users/f02ff01acf35478ba73c99a1adbcc005 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 11:12:06.779146 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541aeb3f-8641-4be7-a244-c2f581c7fb99 None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:12:06.779628 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541aeb3f-8641-4be7-a244-c2f581c7fb99 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:12:06.780008 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541aeb3f-8641-4be7-a244-c2f581c7fb99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:12:06.799323 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-541aeb3f-8641-4be7-a244-c2f581c7fb99 None None] MFA Rules not processed for user `ca155a48e2024c32af3fd2e19e5fac11`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 11:12:06.800540 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-541aeb3f-8641-4be7-a244-c2f581c7fb99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:12:06.861600 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1249/2515] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 11:12:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 11:12:06.871453 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49cfd5c0-95ae-4068-895d-e0eaf8097550 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:12:06.873252 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-49cfd5c0-95ae-4068-895d-e0eaf8097550 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:12:06.935338 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49cfd5c0-95ae-4068-895d-e0eaf8097550 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:12:06.939669 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18760297-2de0-414e-bf6a-3e8ecc1283b4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91da2acd2534423091537e1ba068d2b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e30ccf0c1204727b786876c141dbcd2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:12:06.940304 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18760297-2de0-414e-bf6a-3e8ecc1283b4 service neutron] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:12:06.940578 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18760297-2de0-414e-bf6a-3e8ecc1283b4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:12:06.940860 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18760297-2de0-414e-bf6a-3e8ecc1283b4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:12:06.951389 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18760297-2de0-414e-bf6a-3e8ecc1283b4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:12:06.953013 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18760297-2de0-414e-bf6a-3e8ecc1283b4 service neutron] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:12:06.963181 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1267/2516] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 11:12:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 11:12:07.240540 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541aeb3f-8641-4be7-a244-c2f581c7fb99 None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:12:07.241998 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-541aeb3f-8641-4be7-a244-c2f581c7fb99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:12:07.257522 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541aeb3f-8641-4be7-a244-c2f581c7fb99 None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:12:07.262324 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ae2f90-7750-44c8-828f-001ede71cf64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:12:07.264527 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ae2f90-7750-44c8-828f-001ede71cf64 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:12:07.264815 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ae2f90-7750-44c8-828f-001ede71cf64 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:12:07.267552 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ae2f90-7750-44c8-828f-001ede71cf64 admin admin] PATH_INFO: `/v3/projects/1e54372406304894bb99817f57aab09d` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:12:07.267552 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ae2f90-7750-44c8-828f-001ede71cf64 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1e54372406304894bb99817f57aab09d)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:12:07.268637 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ae2f90-7750-44c8-828f-001ede71cf64 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:12:07.320299 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-e3ae2f90-7750-44c8-828f-001ede71cf64 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1e54372406304894bb99817f57aab09d', 'request_id': 'req-e3ae2f90-7750-44c8-828f-001ede71cf64'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:12:07.327103 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-e3ae2f90-7750-44c8-828f-001ede71cf64 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1e54372406304894bb99817f57aab09d', 'request_id': 'req-e3ae2f90-7750-44c8-828f-001ede71cf64'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:12:07.332697 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-e3ae2f90-7750-44c8-828f-001ede71cf64 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1e54372406304894bb99817f57aab09d', 'request_id': 'req-e3ae2f90-7750-44c8-828f-001ede71cf64'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:12:07.349737 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-e3ae2f90-7750-44c8-828f-001ede71cf64 admin admin] The token cache is being invalidate because project 1e54372406304894bb99817f57aab09d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81423) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 11:12:07.350050 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-e3ae2f90-7750-44c8-828f-001ede71cf64 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:12:07.350705 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-e3ae2f90-7750-44c8-828f-001ede71cf64 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:12:07.352213 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1250/2517] 10.222.0.40 () {58 vars in 1233 bytes} [Fri Aug 26 11:12:07 2022] DELETE /identity/v3/projects/1e54372406304894bb99817f57aab09d => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 11:12:07.357173 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18760297-2de0-414e-bf6a-3e8ecc1283b4 service neutron] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:12:07.367265 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18760297-2de0-414e-bf6a-3e8ecc1283b4 service neutron] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:12:07.425338 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99949d27-cff2-4149-8ca2-98532109446c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:12:07.426320 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99949d27-cff2-4149-8ca2-98532109446c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:12:07.426621 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99949d27-cff2-4149-8ca2-98532109446c admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:12:07.426893 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99949d27-cff2-4149-8ca2-98532109446c admin admin] PATH_INFO: `/v3/users/aa61400617e44a419668ed25a9b0dd8f` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:12:07.428043 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99949d27-cff2-4149-8ca2-98532109446c admin admin] RBAC: Authorizing `identity:delete_user(user_id=aa61400617e44a419668ed25a9b0dd8f)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:12:07.430013 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99949d27-cff2-4149-8ca2-98532109446c admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:12:07.477707 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-99949d27-cff2-4149-8ca2-98532109446c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'aa61400617e44a419668ed25a9b0dd8f'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:12:07.478369 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-99949d27-cff2-4149-8ca2-98532109446c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'aa61400617e44a419668ed25a9b0dd8f'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:12:07.486308 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-99949d27-cff2-4149-8ca2-98532109446c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'aa61400617e44a419668ed25a9b0dd8f'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:12:07.486921 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-99949d27-cff2-4149-8ca2-98532109446c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'aa61400617e44a419668ed25a9b0dd8f'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:12:07.500428 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-99949d27-cff2-4149-8ca2-98532109446c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'aa61400617e44a419668ed25a9b0dd8f'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:12:07.510061 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 11:12:07.516955 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1268/2518] 10.222.0.40 () {58 vars in 1224 bytes} [Fri Aug 26 11:12:07 2022] DELETE /identity/v3/users/aa61400617e44a419668ed25a9b0dd8f => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 11:12:07.823824 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ae2f90-7750-44c8-828f-001ede71cf64 admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:12:07.825922 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-e3ae2f90-7750-44c8-828f-001ede71cf64 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:12:07.839582 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ae2f90-7750-44c8-828f-001ede71cf64 admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:12:07.885270 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 11:12:07.919535 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9207f285-a501-4879-a77f-3081aa0a0259 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:12:07.920826 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9207f285-a501-4879-a77f-3081aa0a0259 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:12:07.921233 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9207f285-a501-4879-a77f-3081aa0a0259 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:12:07.921542 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9207f285-a501-4879-a77f-3081aa0a0259 admin admin] PATH_INFO: `/v3/projects/121d89e3e8594749a4398243ea1ae3ff` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:12:07.922901 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9207f285-a501-4879-a77f-3081aa0a0259 admin admin] RBAC: Authorizing `identity:delete_project(project_id=121d89e3e8594749a4398243ea1ae3ff)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:12:07.924840 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9207f285-a501-4879-a77f-3081aa0a0259 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:12:07.977385 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-9207f285-a501-4879-a77f-3081aa0a0259 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '121d89e3e8594749a4398243ea1ae3ff', 'request_id': 'req-9207f285-a501-4879-a77f-3081aa0a0259'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:12:07.986556 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-9207f285-a501-4879-a77f-3081aa0a0259 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '121d89e3e8594749a4398243ea1ae3ff', 'request_id': 'req-9207f285-a501-4879-a77f-3081aa0a0259'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:12:07.992892 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-9207f285-a501-4879-a77f-3081aa0a0259 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '121d89e3e8594749a4398243ea1ae3ff', 'request_id': 'req-9207f285-a501-4879-a77f-3081aa0a0259'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:12:08.009821 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-9207f285-a501-4879-a77f-3081aa0a0259 admin admin] The token cache is being invalidate because project 121d89e3e8594749a4398243ea1ae3ff was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81423) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 11:12:08.010457 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-9207f285-a501-4879-a77f-3081aa0a0259 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:12:08.011653 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-9207f285-a501-4879-a77f-3081aa0a0259 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:12:08.014302 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1251/2519] 10.222.0.40 () {58 vars in 1233 bytes} [Fri Aug 26 11:12:07 2022] DELETE /identity/v3/projects/121d89e3e8594749a4398243ea1ae3ff => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 11:12:08.019196 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0425da5a-b3dd-4a81-bafb-a5b473317655 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:12:08.019682 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0425da5a-b3dd-4a81-bafb-a5b473317655 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:12:08.019809 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0425da5a-b3dd-4a81-bafb-a5b473317655 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:12:08.040767 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-0425da5a-b3dd-4a81-bafb-a5b473317655 None None] MFA Rules not processed for user `ca155a48e2024c32af3fd2e19e5fac11`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 11:12:08.042051 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-0425da5a-b3dd-4a81-bafb-a5b473317655 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:12:08.103730 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1269/2520] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 11:12:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 11:12:08.112578 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a43503-4d7c-45fb-9d58-5b3c1d62c68e None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:12:08.112861 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a43503-4d7c-45fb-9d58-5b3c1d62c68e None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:12:08.113134 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a43503-4d7c-45fb-9d58-5b3c1d62c68e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:12:08.134095 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-f3a43503-4d7c-45fb-9d58-5b3c1d62c68e None None] MFA Rules not processed for user `ca155a48e2024c32af3fd2e19e5fac11`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 11:12:08.135353 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-f3a43503-4d7c-45fb-9d58-5b3c1d62c68e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:12:08.144226 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1252/2521] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 11:12:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 11:12:08.148534 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0425da5a-b3dd-4a81-bafb-a5b473317655 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:12:08.159462 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0425da5a-b3dd-4a81-bafb-a5b473317655 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:12:08.164219 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773ab696-c691-4e8d-ab42-14d23ebb9cc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:12:08.164856 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773ab696-c691-4e8d-ab42-14d23ebb9cc2 admin admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:12:08.165139 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773ab696-c691-4e8d-ab42-14d23ebb9cc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:12:08.165411 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773ab696-c691-4e8d-ab42-14d23ebb9cc2 admin admin] PATH_INFO: `/v3/domains` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:12:08.166095 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-773ab696-c691-4e8d-ab42-14d23ebb9cc2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 11:12:08.166397 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-773ab696-c691-4e8d-ab42-14d23ebb9cc2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:12:08.167997 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-773ab696-c691-4e8d-ab42-14d23ebb9cc2 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:12:08.180190 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1270/2522] 10.222.0.40 () {58 vars in 1153 bytes} [Fri Aug 26 11:12:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 11:12:08.184787 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a43503-4d7c-45fb-9d58-5b3c1d62c68e None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:12:08.194539 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a43503-4d7c-45fb-9d58-5b3c1d62c68e None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:12:08.199236 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a92a3c6-fdd1-485b-aa68-9446010f3de1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:12:08.199997 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a92a3c6-fdd1-485b-aa68-9446010f3de1 admin admin] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:12:08.201081 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a92a3c6-fdd1-485b-aa68-9446010f3de1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:12:08.201369 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a92a3c6-fdd1-485b-aa68-9446010f3de1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:12:08.202059 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a92a3c6-fdd1-485b-aa68-9446010f3de1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:12:08.203981 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a92a3c6-fdd1-485b-aa68-9446010f3de1 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:12:08.232798 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1253/2523] 10.222.0.40 () {60 vars in 1153 bytes} [Fri Aug 26 11:12:08 2022] POST /identity/v3/projects => generated 385 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 26 11:12:08.237217 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773ab696-c691-4e8d-ab42-14d23ebb9cc2 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:12:08.248039 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773ab696-c691-4e8d-ab42-14d23ebb9cc2 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:12:08.306148 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def6faa3-f668-4cb7-bdb3-1d3750e98967 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:12:08.306764 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def6faa3-f668-4cb7-bdb3-1d3750e98967 admin admin] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:12:08.307045 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def6faa3-f668-4cb7-bdb3-1d3750e98967 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:12:08.307315 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def6faa3-f668-4cb7-bdb3-1d3750e98967 admin admin] PATH_INFO: `/v3/users` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:12:08.307941 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-def6faa3-f668-4cb7-bdb3-1d3750e98967 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:12:08.310124 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-def6faa3-f668-4cb7-bdb3-1d3750e98967 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:12:08.332167 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1271/2524] 10.222.0.40 () {60 vars in 1144 bytes} [Fri Aug 26 11:12:08 2022] POST /identity/v3/users => generated 293 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 11:12:08.336515 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a92a3c6-fdd1-485b-aa68-9446010f3de1 admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:12:08.349628 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a92a3c6-fdd1-485b-aa68-9446010f3de1 admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:12:08.354448 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89f7daf-cffa-4920-97c3-c3d39ec0673d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:12:08.355056 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89f7daf-cffa-4920-97c3-c3d39ec0673d admin admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:12:08.355329 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89f7daf-cffa-4920-97c3-c3d39ec0673d admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:12:08.355636 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89f7daf-cffa-4920-97c3-c3d39ec0673d admin admin] PATH_INFO: `/v3/roles` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:12:08.356261 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d89f7daf-cffa-4920-97c3-c3d39ec0673d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:12:08.357906 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d89f7daf-cffa-4920-97c3-c3d39ec0673d admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:12:08.369034 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1254/2525] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 26 11:12:08 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 26 11:12:08.373392 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def6faa3-f668-4cb7-bdb3-1d3750e98967 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:12:08.384640 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def6faa3-f668-4cb7-bdb3-1d3750e98967 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:12:08.390094 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccdc30f2-d1f9-4e66-877e-4149e2ccc9de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:12:08.390701 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccdc30f2-d1f9-4e66-877e-4149e2ccc9de admin admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:12:08.390977 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccdc30f2-d1f9-4e66-877e-4149e2ccc9de admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:12:08.391248 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccdc30f2-d1f9-4e66-877e-4149e2ccc9de admin admin] PATH_INFO: `/v3/roles` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:12:08.391884 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccdc30f2-d1f9-4e66-877e-4149e2ccc9de admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:12:08.393491 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccdc30f2-d1f9-4e66-877e-4149e2ccc9de admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:12:08.405706 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1272/2526] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 26 11:12:08 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 11:12:08.409732 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89f7daf-cffa-4920-97c3-c3d39ec0673d admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:12:08.420191 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89f7daf-cffa-4920-97c3-c3d39ec0673d admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:12:08.424869 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee9c7e0-01f4-406d-b65e-2c5315b901ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:12:08.425524 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee9c7e0-01f4-406d-b65e-2c5315b901ea admin admin] REQUEST_METHOD: `PUT` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:12:08.425794 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee9c7e0-01f4-406d-b65e-2c5315b901ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:12:08.426096 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee9c7e0-01f4-406d-b65e-2c5315b901ea admin admin] PATH_INFO: `/v3/projects/e5a1155592474f31b4a40af1a9ef50a5/users/078f95ada9a4485a9e0d20c2783a1814/roles/23e174ce743b46eabe618c212691da55` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:12:08.441793 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee9c7e0-01f4-406d-b65e-2c5315b901ea admin admin] RBAC: Authorizing `identity:create_grant(project_id=e5a1155592474f31b4a40af1a9ef50a5, user_id=078f95ada9a4485a9e0d20c2783a1814, role_id=23e174ce743b46eabe618c212691da55)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:12:08.444475 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee9c7e0-01f4-406d-b65e-2c5315b901ea admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:12:08.458513 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1255/2527] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 26 11:12:08 2022] PUT /identity/v3/projects/e5a1155592474f31b4a40af1a9ef50a5/users/078f95ada9a4485a9e0d20c2783a1814/roles/23e174ce743b46eabe618c212691da55 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 11:12:08.470713 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382f5e48-8063-452f-9584-06a7a637ee5f None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:12:08.470991 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382f5e48-8063-452f-9584-06a7a637ee5f None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:12:08.471264 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382f5e48-8063-452f-9584-06a7a637ee5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:12:08.500819 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-382f5e48-8063-452f-9584-06a7a637ee5f None None] MFA Rules not processed for user `078f95ada9a4485a9e0d20c2783a1814`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 11:12:08.501968 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-382f5e48-8063-452f-9584-06a7a637ee5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:12:08.558219 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 11:12:08.572046 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1273/2528] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 11:12:08 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 11:12:08.577292 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee9c7e0-01f4-406d-b65e-2c5315b901ea admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:12:08.636579 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 11:12:08.641404 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee9c7e0-01f4-406d-b65e-2c5315b901ea admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:12:08.648145 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5a841b-4687-4ea8-bfd7-6eb16d33e565 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:12:08.648992 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5a841b-4687-4ea8-bfd7-6eb16d33e565 admin admin] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:12:08.649450 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5a841b-4687-4ea8-bfd7-6eb16d33e565 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:12:08.649815 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5a841b-4687-4ea8-bfd7-6eb16d33e565 admin admin] PATH_INFO: `/v3/projects` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:12:08.650637 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be5a841b-4687-4ea8-bfd7-6eb16d33e565 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:12:08.652591 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be5a841b-4687-4ea8-bfd7-6eb16d33e565 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:12:08.680080 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1256/2529] 10.222.0.40 () {60 vars in 1153 bytes} [Fri Aug 26 11:12:08 2022] POST /identity/v3/projects => generated 387 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 26 11:12:08.686874 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382f5e48-8063-452f-9584-06a7a637ee5f None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:12:08.698920 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382f5e48-8063-452f-9584-06a7a637ee5f None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:12:08.758337 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b26148-44ed-4290-948f-170764b0015d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:12:08.758895 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b26148-44ed-4290-948f-170764b0015d admin admin] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:12:08.759176 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b26148-44ed-4290-948f-170764b0015d admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:12:08.759466 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b26148-44ed-4290-948f-170764b0015d admin admin] PATH_INFO: `/v3/users` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:12:08.760088 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b26148-44ed-4290-948f-170764b0015d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:12:08.762050 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b26148-44ed-4290-948f-170764b0015d admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:12:08.784722 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1274/2530] 10.222.0.40 () {60 vars in 1144 bytes} [Fri Aug 26 11:12:08 2022] POST /identity/v3/users => generated 300 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 26 11:12:08.788577 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5a841b-4687-4ea8-bfd7-6eb16d33e565 admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:12:08.799635 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5a841b-4687-4ea8-bfd7-6eb16d33e565 admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:12:08.805015 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf540908-87fc-4ad6-bda6-2acad808fdc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:12:08.805720 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf540908-87fc-4ad6-bda6-2acad808fdc5 admin admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:12:08.806029 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf540908-87fc-4ad6-bda6-2acad808fdc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:12:08.806299 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf540908-87fc-4ad6-bda6-2acad808fdc5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:12:08.806944 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf540908-87fc-4ad6-bda6-2acad808fdc5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:12:08.808566 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf540908-87fc-4ad6-bda6-2acad808fdc5 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:12:08.824386 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1257/2531] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 26 11:12:08 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 11:12:08.826597 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b26148-44ed-4290-948f-170764b0015d admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:12:08.837657 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b26148-44ed-4290-948f-170764b0015d admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:12:08.848352 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685b1082-86e0-4a9f-b8d2-9f4cba887195 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:12:08.849035 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685b1082-86e0-4a9f-b8d2-9f4cba887195 admin admin] REQUEST_METHOD: `PUT` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:12:08.849312 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685b1082-86e0-4a9f-b8d2-9f4cba887195 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:12:08.849590 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685b1082-86e0-4a9f-b8d2-9f4cba887195 admin admin] PATH_INFO: `/v3/projects/49f520b279884a2eaaa52f739d42f9ec/users/23ce6be2077b43549e00253a5d967cf0/roles/1f97e4602e2e4d6498913a81f9ae8d6a` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:12:08.866921 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-685b1082-86e0-4a9f-b8d2-9f4cba887195 admin admin] RBAC: Authorizing `identity:create_grant(project_id=49f520b279884a2eaaa52f739d42f9ec, user_id=23ce6be2077b43549e00253a5d967cf0, role_id=1f97e4602e2e4d6498913a81f9ae8d6a)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:12:08.869674 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-685b1082-86e0-4a9f-b8d2-9f4cba887195 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:12:08.888270 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1275/2532] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 26 11:12:08 2022] PUT /identity/v3/projects/49f520b279884a2eaaa52f739d42f9ec/users/23ce6be2077b43549e00253a5d967cf0/roles/1f97e4602e2e4d6498913a81f9ae8d6a => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 11:12:08.893052 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf540908-87fc-4ad6-bda6-2acad808fdc5 admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:12:08.898154 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685b1082-86e0-4a9f-b8d2-9f4cba887195 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:12:08.901118 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-685b1082-86e0-4a9f-b8d2-9f4cba887195 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:12:08.907281 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf540908-87fc-4ad6-bda6-2acad808fdc5 admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:12:08.966674 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685b1082-86e0-4a9f-b8d2-9f4cba887195 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:12:08.970546 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba656f0-fd0c-4f8a-9056-42b90662df73 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91da2acd2534423091537e1ba068d2b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e30ccf0c1204727b786876c141dbcd2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:12:08.970734 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e11f73-8bea-4cc1-8cd3-667a69d95764 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:12:08.971159 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba656f0-fd0c-4f8a-9056-42b90662df73 service neutron] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:12:08.971267 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e11f73-8bea-4cc1-8cd3-667a69d95764 admin admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:12:08.971432 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba656f0-fd0c-4f8a-9056-42b90662df73 service neutron] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:12:08.971534 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e11f73-8bea-4cc1-8cd3-667a69d95764 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:12:08.971698 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba656f0-fd0c-4f8a-9056-42b90662df73 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:12:08.971804 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e11f73-8bea-4cc1-8cd3-667a69d95764 admin admin] PATH_INFO: `/v3/roles` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:12:08.972364 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e11f73-8bea-4cc1-8cd3-667a69d95764 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:12:08.973477 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-0ba656f0-fd0c-4f8a-9056-42b90662df73 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:12:08.974518 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e11f73-8bea-4cc1-8cd3-667a69d95764 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:12:08.987487 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba656f0-fd0c-4f8a-9056-42b90662df73 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:12:08.987713 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1258/2533] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 26 11:12:08 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 11:12:08.988949 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba656f0-fd0c-4f8a-9056-42b90662df73 service neutron] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:12:08.991749 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e11f73-8bea-4cc1-8cd3-667a69d95764 admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:12:09.000066 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1276/2534] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 11:12:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 26 11:12:09.004066 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e11f73-8bea-4cc1-8cd3-667a69d95764 admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:12:09.009853 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0569b88-0466-4d97-93fc-737d327c5cc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:12:09.010476 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0569b88-0466-4d97-93fc-737d327c5cc2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:12:09.010750 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0569b88-0466-4d97-93fc-737d327c5cc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:12:09.011018 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0569b88-0466-4d97-93fc-737d327c5cc2 admin admin] PATH_INFO: `/v3/projects/49f520b279884a2eaaa52f739d42f9ec/users/23ce6be2077b43549e00253a5d967cf0/roles/1f97e4602e2e4d6498913a81f9ae8d6a` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:12:09.013598 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0569b88-0466-4d97-93fc-737d327c5cc2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=49f520b279884a2eaaa52f739d42f9ec, user_id=23ce6be2077b43549e00253a5d967cf0, role_id=1f97e4602e2e4d6498913a81f9ae8d6a)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:12:09.016314 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0569b88-0466-4d97-93fc-737d327c5cc2 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:12:09.030160 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1259/2535] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 26 11:12:08 2022] PUT /identity/v3/projects/49f520b279884a2eaaa52f739d42f9ec/users/23ce6be2077b43549e00253a5d967cf0/roles/1f97e4602e2e4d6498913a81f9ae8d6a => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 26 11:12:09.040577 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329fa1c7-de3a-456a-ab75-d2c74dafa58a None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:12:09.041130 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329fa1c7-de3a-456a-ab75-d2c74dafa58a None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:12:09.041246 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329fa1c7-de3a-456a-ab75-d2c74dafa58a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:12:09.081851 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-329fa1c7-de3a-456a-ab75-d2c74dafa58a None None] MFA Rules not processed for user `23ce6be2077b43549e00253a5d967cf0`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 11:12:09.083128 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-329fa1c7-de3a-456a-ab75-d2c74dafa58a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:12:09.136979 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 11:12:09.163278 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1277/2536] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 11:12:09 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 11:12:09.175311 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0569b88-0466-4d97-93fc-737d327c5cc2 admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:12:09.177017 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-e0569b88-0466-4d97-93fc-737d327c5cc2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:12:09.244073 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0569b88-0466-4d97-93fc-737d327c5cc2 admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:12:09.248874 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17289d7a-f884-4247-b37f-3735c00fd4fe service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c277338a7f814cc6b01c5419775031bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e30ccf0c1204727b786876c141dbcd2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:12:09.249493 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17289d7a-f884-4247-b37f-3735c00fd4fe service cinder] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:12:09.249770 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17289d7a-f884-4247-b37f-3735c00fd4fe service cinder] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:12:09.250101 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17289d7a-f884-4247-b37f-3735c00fd4fe service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:12:09.260339 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17289d7a-f884-4247-b37f-3735c00fd4fe service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:12:09.261687 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17289d7a-f884-4247-b37f-3735c00fd4fe service cinder] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:12:09.272874 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1260/2537] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 11:12:09 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 26 11:12:09.353236 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329fa1c7-de3a-456a-ab75-d2c74dafa58a None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:12:09.421663 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329fa1c7-de3a-456a-ab75-d2c74dafa58a None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:12:09.426303 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dda4219-c168-427c-a4a6-811d2c75b72e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:12:09.426960 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dda4219-c168-427c-a4a6-811d2c75b72e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:12:09.427233 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dda4219-c168-427c-a4a6-811d2c75b72e admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:12:09.427504 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dda4219-c168-427c-a4a6-811d2c75b72e admin admin] PATH_INFO: `/v3/users/078f95ada9a4485a9e0d20c2783a1814` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:12:09.428682 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dda4219-c168-427c-a4a6-811d2c75b72e admin admin] RBAC: Authorizing `identity:delete_user(user_id=078f95ada9a4485a9e0d20c2783a1814)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:12:09.430693 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dda4219-c168-427c-a4a6-811d2c75b72e admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:12:09.485371 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-7dda4219-c168-427c-a4a6-811d2c75b72e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '078f95ada9a4485a9e0d20c2783a1814'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:12:09.486248 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-7dda4219-c168-427c-a4a6-811d2c75b72e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '078f95ada9a4485a9e0d20c2783a1814'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:12:09.493763 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-7dda4219-c168-427c-a4a6-811d2c75b72e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '078f95ada9a4485a9e0d20c2783a1814'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:12:09.494512 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-7dda4219-c168-427c-a4a6-811d2c75b72e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '078f95ada9a4485a9e0d20c2783a1814'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:12:09.509365 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-7dda4219-c168-427c-a4a6-811d2c75b72e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '078f95ada9a4485a9e0d20c2783a1814'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:12:09.527341 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1278/2538] 10.222.0.40 () {58 vars in 1224 bytes} [Fri Aug 26 11:12:09 2022] DELETE /identity/v3/users/078f95ada9a4485a9e0d20c2783a1814 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 11:12:09.532670 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51259aad-6d34-4225-86a7-941c06e1331d None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:12:09.532957 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51259aad-6d34-4225-86a7-941c06e1331d None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:12:09.533227 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51259aad-6d34-4225-86a7-941c06e1331d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:12:09.553821 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-51259aad-6d34-4225-86a7-941c06e1331d None None] MFA Rules not processed for user `ca155a48e2024c32af3fd2e19e5fac11`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 11:12:09.555082 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-51259aad-6d34-4225-86a7-941c06e1331d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:12:09.616936 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 11:12:09.628148 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1261/2539] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 11:12:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 11:12:09.638648 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dda4219-c168-427c-a4a6-811d2c75b72e admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:12:09.640165 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-7dda4219-c168-427c-a4a6-811d2c75b72e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:12:09.706972 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dda4219-c168-427c-a4a6-811d2c75b72e admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:12:09.712417 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef4bab2-070f-4fc8-b696-b894c4a17529 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91da2acd2534423091537e1ba068d2b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e30ccf0c1204727b786876c141dbcd2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:12:09.713126 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef4bab2-070f-4fc8-b696-b894c4a17529 service neutron] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:12:09.713437 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef4bab2-070f-4fc8-b696-b894c4a17529 service neutron] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:12:09.713712 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef4bab2-070f-4fc8-b696-b894c4a17529 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:12:09.724474 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef4bab2-070f-4fc8-b696-b894c4a17529 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:12:09.725924 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef4bab2-070f-4fc8-b696-b894c4a17529 service neutron] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:12:09.736396 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1279/2540] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 11:12:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 11:12:10.259090 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51259aad-6d34-4225-86a7-941c06e1331d None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:12:10.260584 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-51259aad-6d34-4225-86a7-941c06e1331d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:12:10.278348 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51259aad-6d34-4225-86a7-941c06e1331d None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:12:10.284387 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17556bb9-9440-403d-b1b8-3b10cf3ad55f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:12:10.285062 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17556bb9-9440-403d-b1b8-3b10cf3ad55f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:12:10.285380 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17556bb9-9440-403d-b1b8-3b10cf3ad55f admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:12:10.285618 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17556bb9-9440-403d-b1b8-3b10cf3ad55f admin admin] PATH_INFO: `/v3/projects/e5a1155592474f31b4a40af1a9ef50a5` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:12:10.286881 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17556bb9-9440-403d-b1b8-3b10cf3ad55f admin admin] RBAC: Authorizing `identity:delete_project(project_id=e5a1155592474f31b4a40af1a9ef50a5)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:12:10.288825 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17556bb9-9440-403d-b1b8-3b10cf3ad55f admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:12:10.345095 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-17556bb9-9440-403d-b1b8-3b10cf3ad55f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e5a1155592474f31b4a40af1a9ef50a5', 'request_id': 'req-17556bb9-9440-403d-b1b8-3b10cf3ad55f'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:12:10.352354 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-17556bb9-9440-403d-b1b8-3b10cf3ad55f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e5a1155592474f31b4a40af1a9ef50a5', 'request_id': 'req-17556bb9-9440-403d-b1b8-3b10cf3ad55f'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:12:10.359834 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-17556bb9-9440-403d-b1b8-3b10cf3ad55f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e5a1155592474f31b4a40af1a9ef50a5', 'request_id': 'req-17556bb9-9440-403d-b1b8-3b10cf3ad55f'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:12:10.377217 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-17556bb9-9440-403d-b1b8-3b10cf3ad55f admin admin] The token cache is being invalidate because project e5a1155592474f31b4a40af1a9ef50a5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81423) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 11:12:10.377507 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-17556bb9-9440-403d-b1b8-3b10cf3ad55f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:12:10.378231 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-17556bb9-9440-403d-b1b8-3b10cf3ad55f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:12:10.379767 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1262/2541] 10.222.0.40 () {58 vars in 1233 bytes} [Fri Aug 26 11:12:10 2022] DELETE /identity/v3/projects/e5a1155592474f31b4a40af1a9ef50a5 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 11:12:10.383876 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef4bab2-070f-4fc8-b696-b894c4a17529 service neutron] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:12:10.394197 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef4bab2-070f-4fc8-b696-b894c4a17529 service neutron] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:12:10.400560 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 11:12:10.459381 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107ab88d-bfbc-43bb-a8b3-a48c053710fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:12:10.459996 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107ab88d-bfbc-43bb-a8b3-a48c053710fe admin admin] REQUEST_METHOD: `DELETE` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:12:10.460266 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107ab88d-bfbc-43bb-a8b3-a48c053710fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:12:10.460544 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107ab88d-bfbc-43bb-a8b3-a48c053710fe admin admin] PATH_INFO: `/v3/users/23ce6be2077b43549e00253a5d967cf0` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:12:10.461730 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-107ab88d-bfbc-43bb-a8b3-a48c053710fe admin admin] RBAC: Authorizing `identity:delete_user(user_id=23ce6be2077b43549e00253a5d967cf0)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:12:10.463713 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-107ab88d-bfbc-43bb-a8b3-a48c053710fe admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:12:10.518658 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-107ab88d-bfbc-43bb-a8b3-a48c053710fe admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '23ce6be2077b43549e00253a5d967cf0'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:12:10.519329 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-107ab88d-bfbc-43bb-a8b3-a48c053710fe admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '23ce6be2077b43549e00253a5d967cf0'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:12:10.526448 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-107ab88d-bfbc-43bb-a8b3-a48c053710fe admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '23ce6be2077b43549e00253a5d967cf0'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:12:10.527172 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-107ab88d-bfbc-43bb-a8b3-a48c053710fe admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '23ce6be2077b43549e00253a5d967cf0'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:12:10.542131 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-107ab88d-bfbc-43bb-a8b3-a48c053710fe admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '23ce6be2077b43549e00253a5d967cf0'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:12:10.557665 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1280/2542] 10.222.0.40 () {58 vars in 1224 bytes} [Fri Aug 26 11:12:10 2022] DELETE /identity/v3/users/23ce6be2077b43549e00253a5d967cf0 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 11:12:11.043006 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17556bb9-9440-403d-b1b8-3b10cf3ad55f admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:12:11.044773 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-17556bb9-9440-403d-b1b8-3b10cf3ad55f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:12:11.059334 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17556bb9-9440-403d-b1b8-3b10cf3ad55f admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:12:11.133223 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2abac240-71d8-4014-b0d5-2405952df331 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:12:11.134079 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abac240-71d8-4014-b0d5-2405952df331 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:12:11.134549 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abac240-71d8-4014-b0d5-2405952df331 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:12:11.134926 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abac240-71d8-4014-b0d5-2405952df331 admin admin] PATH_INFO: `/v3/projects/49f520b279884a2eaaa52f739d42f9ec` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:12:11.136320 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2abac240-71d8-4014-b0d5-2405952df331 admin admin] RBAC: Authorizing `identity:delete_project(project_id=49f520b279884a2eaaa52f739d42f9ec)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:12:11.138951 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2abac240-71d8-4014-b0d5-2405952df331 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:12:11.197430 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-2abac240-71d8-4014-b0d5-2405952df331 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '49f520b279884a2eaaa52f739d42f9ec', 'request_id': 'req-2abac240-71d8-4014-b0d5-2405952df331'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:12:11.205048 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-2abac240-71d8-4014-b0d5-2405952df331 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '49f520b279884a2eaaa52f739d42f9ec', 'request_id': 'req-2abac240-71d8-4014-b0d5-2405952df331'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:12:11.212649 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-2abac240-71d8-4014-b0d5-2405952df331 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '49f520b279884a2eaaa52f739d42f9ec', 'request_id': 'req-2abac240-71d8-4014-b0d5-2405952df331'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:12:11.238175 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-2abac240-71d8-4014-b0d5-2405952df331 admin admin] The token cache is being invalidate because project 49f520b279884a2eaaa52f739d42f9ec was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81423) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 11:12:11.238676 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-2abac240-71d8-4014-b0d5-2405952df331 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:12:11.239591 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-2abac240-71d8-4014-b0d5-2405952df331 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:12:11.242394 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1263/2543] 10.222.0.40 () {58 vars in 1233 bytes} [Fri Aug 26 11:12:11 2022] DELETE /identity/v3/projects/49f520b279884a2eaaa52f739d42f9ec => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 11:12:11.247932 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2124d188-64b2-49f5-a7c8-739a688bb2b5 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:12:11.248842 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2124d188-64b2-49f5-a7c8-739a688bb2b5 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:12:11.249133 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2124d188-64b2-49f5-a7c8-739a688bb2b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:12:11.275244 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-2124d188-64b2-49f5-a7c8-739a688bb2b5 None None] MFA Rules not processed for user `ca155a48e2024c32af3fd2e19e5fac11`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 11:12:11.276489 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-2124d188-64b2-49f5-a7c8-739a688bb2b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:12:11.353197 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1281/2544] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 11:12:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 11:12:11.361681 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3cf3d5-6f2b-446a-8222-5e0452cbf465 None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:12:11.362248 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3cf3d5-6f2b-446a-8222-5e0452cbf465 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:12:11.362248 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3cf3d5-6f2b-446a-8222-5e0452cbf465 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:12:11.385169 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-1d3cf3d5-6f2b-446a-8222-5e0452cbf465 None None] MFA Rules not processed for user `ca155a48e2024c32af3fd2e19e5fac11`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 11:12:11.386459 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-1d3cf3d5-6f2b-446a-8222-5e0452cbf465 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:12:11.394995 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1264/2545] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 11:12:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 11:12:11.399606 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2124d188-64b2-49f5-a7c8-739a688bb2b5 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:12:11.411230 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2124d188-64b2-49f5-a7c8-739a688bb2b5 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:12:11.416199 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7674a2-6242-4dee-9b6a-f073c801d27a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:12:11.416916 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7674a2-6242-4dee-9b6a-f073c801d27a admin admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:12:11.417267 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7674a2-6242-4dee-9b6a-f073c801d27a admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:12:11.417606 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7674a2-6242-4dee-9b6a-f073c801d27a admin admin] PATH_INFO: `/v3/domains` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:12:11.418376 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea7674a2-6242-4dee-9b6a-f073c801d27a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 11:12:11.418747 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea7674a2-6242-4dee-9b6a-f073c801d27a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:12:11.421102 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea7674a2-6242-4dee-9b6a-f073c801d27a admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:12:11.433806 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1282/2546] 10.222.0.40 () {58 vars in 1153 bytes} [Fri Aug 26 11:12:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 11:12:11.438635 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3cf3d5-6f2b-446a-8222-5e0452cbf465 None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:12:11.444495 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 11:12:11.449151 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3cf3d5-6f2b-446a-8222-5e0452cbf465 None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:12:11.455708 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6cc674-ad8e-468d-be25-b49a47d89670 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:12:11.456334 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6cc674-ad8e-468d-be25-b49a47d89670 admin admin] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:12:11.456967 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6cc674-ad8e-468d-be25-b49a47d89670 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:12:11.457461 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6cc674-ad8e-468d-be25-b49a47d89670 admin admin] PATH_INFO: `/v3/projects` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:12:11.458027 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d6cc674-ad8e-468d-be25-b49a47d89670 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:12:11.459937 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d6cc674-ad8e-468d-be25-b49a47d89670 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:12:11.491522 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1265/2547] 10.222.0.40 () {60 vars in 1153 bytes} [Fri Aug 26 11:12:11 2022] POST /identity/v3/projects => generated 375 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 11:12:11.496214 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7674a2-6242-4dee-9b6a-f073c801d27a admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:12:11.508115 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7674a2-6242-4dee-9b6a-f073c801d27a admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:12:11.522070 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 11:12:11.586822 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43cb31cb-170b-4782-8558-ade943b259cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:12:11.588511 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43cb31cb-170b-4782-8558-ade943b259cc admin admin] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:12:11.588511 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43cb31cb-170b-4782-8558-ade943b259cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:12:11.588511 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43cb31cb-170b-4782-8558-ade943b259cc admin admin] PATH_INFO: `/v3/users` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:12:11.588927 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43cb31cb-170b-4782-8558-ade943b259cc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:12:11.590705 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43cb31cb-170b-4782-8558-ade943b259cc admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:12:11.621309 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1283/2548] 10.222.0.40 () {60 vars in 1144 bytes} [Fri Aug 26 11:12:11 2022] POST /identity/v3/users => generated 288 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 11:12:11.626135 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6cc674-ad8e-468d-be25-b49a47d89670 admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:12:11.638579 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6cc674-ad8e-468d-be25-b49a47d89670 admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:12:11.643946 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8932d7-8b6b-46ff-92bb-b53f9f2dd968 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:12:11.644679 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8932d7-8b6b-46ff-92bb-b53f9f2dd968 admin admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:12:11.645198 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8932d7-8b6b-46ff-92bb-b53f9f2dd968 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:12:11.645540 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8932d7-8b6b-46ff-92bb-b53f9f2dd968 admin admin] PATH_INFO: `/v3/roles` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:12:11.646336 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b8932d7-8b6b-46ff-92bb-b53f9f2dd968 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:12:11.648050 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b8932d7-8b6b-46ff-92bb-b53f9f2dd968 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:12:11.662042 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1266/2549] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 26 11:12:11 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 26 11:12:11.667735 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43cb31cb-170b-4782-8558-ade943b259cc admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:12:11.683116 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43cb31cb-170b-4782-8558-ade943b259cc admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:12:11.688594 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a26d4f5-5b47-4dfc-8455-1e5ccafa991f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:12:11.689754 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a26d4f5-5b47-4dfc-8455-1e5ccafa991f admin admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:12:11.690104 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a26d4f5-5b47-4dfc-8455-1e5ccafa991f admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:12:11.690382 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a26d4f5-5b47-4dfc-8455-1e5ccafa991f admin admin] PATH_INFO: `/v3/roles` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:12:11.691038 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a26d4f5-5b47-4dfc-8455-1e5ccafa991f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:12:11.693634 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a26d4f5-5b47-4dfc-8455-1e5ccafa991f admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:12:11.718369 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1284/2550] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 26 11:12:11 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 11:12:11.719701 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8932d7-8b6b-46ff-92bb-b53f9f2dd968 admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:12:11.733247 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8932d7-8b6b-46ff-92bb-b53f9f2dd968 admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:12:11.738223 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621a1474-3e05-465d-9991-a10a729c4da6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:12:11.738902 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621a1474-3e05-465d-9991-a10a729c4da6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:12:11.739194 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621a1474-3e05-465d-9991-a10a729c4da6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:12:11.739463 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621a1474-3e05-465d-9991-a10a729c4da6 admin admin] PATH_INFO: `/v3/projects/8ab915955b874605b29b06a9ac964778/users/0c023a6b12d24c138d104a2a4e8b02f5/roles/23e174ce743b46eabe618c212691da55` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:12:11.757110 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-621a1474-3e05-465d-9991-a10a729c4da6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8ab915955b874605b29b06a9ac964778, user_id=0c023a6b12d24c138d104a2a4e8b02f5, role_id=23e174ce743b46eabe618c212691da55)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:12:11.759799 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-621a1474-3e05-465d-9991-a10a729c4da6 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:12:11.777423 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1267/2551] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 26 11:12:11 2022] PUT /identity/v3/projects/8ab915955b874605b29b06a9ac964778/users/0c023a6b12d24c138d104a2a4e8b02f5/roles/23e174ce743b46eabe618c212691da55 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 11:12:11.792007 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b403e83d-ef79-4202-ba64-b5353620b563 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:12:11.792375 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b403e83d-ef79-4202-ba64-b5353620b563 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:12:11.792745 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b403e83d-ef79-4202-ba64-b5353620b563 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:12:11.826117 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-b403e83d-ef79-4202-ba64-b5353620b563 None None] MFA Rules not processed for user `0c023a6b12d24c138d104a2a4e8b02f5`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 11:12:11.827386 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-b403e83d-ef79-4202-ba64-b5353620b563 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:12:11.906642 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1285/2552] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 11:12:11 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 11:12:11.917850 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621a1474-3e05-465d-9991-a10a729c4da6 admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:12:11.919714 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-621a1474-3e05-465d-9991-a10a729c4da6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:12:11.985660 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621a1474-3e05-465d-9991-a10a729c4da6 admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:12:11.989565 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc081d5-9dd5-4673-8363-34a385ef5977 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8631d7ecfd2849daa8694aecccd22a05', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e30ccf0c1204727b786876c141dbcd2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:12:11.990203 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc081d5-9dd5-4673-8363-34a385ef5977 service glance] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:12:11.990486 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc081d5-9dd5-4673-8363-34a385ef5977 service glance] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:12:11.990756 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc081d5-9dd5-4673-8363-34a385ef5977 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:12:12.003367 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc081d5-9dd5-4673-8363-34a385ef5977 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:12:12.005377 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc081d5-9dd5-4673-8363-34a385ef5977 service glance] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:12:12.020233 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1268/2553] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 11:12:11 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 11:12:12.834537 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b403e83d-ef79-4202-ba64-b5353620b563 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:12:12.836177 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-b403e83d-ef79-4202-ba64-b5353620b563 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:12:12.923023 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b403e83d-ef79-4202-ba64-b5353620b563 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:12:12.927647 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0f5d29-cbda-44b2-a4b3-1b99859ebf13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:12:12.928317 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0f5d29-cbda-44b2-a4b3-1b99859ebf13 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:12:12.928585 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0f5d29-cbda-44b2-a4b3-1b99859ebf13 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:12:12.928880 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0f5d29-cbda-44b2-a4b3-1b99859ebf13 admin admin] PATH_INFO: `/v3/users/aedac3f62dd94f87b968f6c71d68a0a3` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:12:12.929999 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad0f5d29-cbda-44b2-a4b3-1b99859ebf13 admin admin] RBAC: Authorizing `identity:delete_user(user_id=aedac3f62dd94f87b968f6c71d68a0a3)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:12:12.931918 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad0f5d29-cbda-44b2-a4b3-1b99859ebf13 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:12:12.964358 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=93,keystone:INSERT=3,keystone:DELETE=4 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 11:12:12.991808 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-ad0f5d29-cbda-44b2-a4b3-1b99859ebf13 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'aedac3f62dd94f87b968f6c71d68a0a3'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:12:12.992587 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-ad0f5d29-cbda-44b2-a4b3-1b99859ebf13 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'aedac3f62dd94f87b968f6c71d68a0a3'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:12:13.001269 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-ad0f5d29-cbda-44b2-a4b3-1b99859ebf13 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'aedac3f62dd94f87b968f6c71d68a0a3'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:12:13.002260 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-ad0f5d29-cbda-44b2-a4b3-1b99859ebf13 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'aedac3f62dd94f87b968f6c71d68a0a3'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:12:13.020067 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-ad0f5d29-cbda-44b2-a4b3-1b99859ebf13 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'aedac3f62dd94f87b968f6c71d68a0a3'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:12:13.039877 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1286/2554] 10.222.0.40 () {58 vars in 1224 bytes} [Fri Aug 26 11:12:12 2022] DELETE /identity/v3/users/aedac3f62dd94f87b968f6c71d68a0a3 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 11:12:13.238981 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc081d5-9dd5-4673-8363-34a385ef5977 service glance] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:12:13.251882 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc081d5-9dd5-4673-8363-34a385ef5977 service glance] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:12:13.314220 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43240549-7d6a-4ee1-9ac2-2447c1c95365 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:12:13.314863 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43240549-7d6a-4ee1-9ac2-2447c1c95365 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:12:13.315152 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43240549-7d6a-4ee1-9ac2-2447c1c95365 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:12:13.315420 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43240549-7d6a-4ee1-9ac2-2447c1c95365 admin admin] PATH_INFO: `/v3/projects/78251e5bdb2548d1bb18aacd4ccbcd34` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:12:13.316643 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43240549-7d6a-4ee1-9ac2-2447c1c95365 admin admin] RBAC: Authorizing `identity:delete_project(project_id=78251e5bdb2548d1bb18aacd4ccbcd34)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:12:13.318615 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43240549-7d6a-4ee1-9ac2-2447c1c95365 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:12:13.374655 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-43240549-7d6a-4ee1-9ac2-2447c1c95365 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '78251e5bdb2548d1bb18aacd4ccbcd34', 'request_id': 'req-43240549-7d6a-4ee1-9ac2-2447c1c95365'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:12:13.376555 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=94,keystone:INSERT=2,keystone:DELETE=4 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 11:12:13.381346 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-43240549-7d6a-4ee1-9ac2-2447c1c95365 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '78251e5bdb2548d1bb18aacd4ccbcd34', 'request_id': 'req-43240549-7d6a-4ee1-9ac2-2447c1c95365'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:12:13.390123 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-43240549-7d6a-4ee1-9ac2-2447c1c95365 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '78251e5bdb2548d1bb18aacd4ccbcd34', 'request_id': 'req-43240549-7d6a-4ee1-9ac2-2447c1c95365'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:12:13.402651 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-43240549-7d6a-4ee1-9ac2-2447c1c95365 admin admin] The token cache is being invalidate because project 78251e5bdb2548d1bb18aacd4ccbcd34 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81423) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 11:12:13.402939 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-43240549-7d6a-4ee1-9ac2-2447c1c95365 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:12:13.403605 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-43240549-7d6a-4ee1-9ac2-2447c1c95365 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:12:13.405110 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1269/2555] 10.222.0.40 () {58 vars in 1233 bytes} [Fri Aug 26 11:12:13 2022] DELETE /identity/v3/projects/78251e5bdb2548d1bb18aacd4ccbcd34 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 11:12:13.410447 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c163cfd6-bc43-4d7f-a6a7-211ebc4e88e8 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:12:13.410738 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c163cfd6-bc43-4d7f-a6a7-211ebc4e88e8 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:12:13.411017 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c163cfd6-bc43-4d7f-a6a7-211ebc4e88e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:12:13.437017 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-c163cfd6-bc43-4d7f-a6a7-211ebc4e88e8 None None] MFA Rules not processed for user `ca155a48e2024c32af3fd2e19e5fac11`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 11:12:13.438529 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-c163cfd6-bc43-4d7f-a6a7-211ebc4e88e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:12:13.503103 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1287/2556] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 11:12:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 11:12:13.512084 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df80275f-cde2-4f52-9bed-663c6edcff1b None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:12:13.512367 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df80275f-cde2-4f52-9bed-663c6edcff1b None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:12:13.513690 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df80275f-cde2-4f52-9bed-663c6edcff1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:12:13.534766 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-df80275f-cde2-4f52-9bed-663c6edcff1b None None] MFA Rules not processed for user `ca155a48e2024c32af3fd2e19e5fac11`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 11:12:13.535989 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-df80275f-cde2-4f52-9bed-663c6edcff1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:12:13.544676 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1270/2557] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 11:12:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 11:12:13.548871 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c163cfd6-bc43-4d7f-a6a7-211ebc4e88e8 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:12:13.560761 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c163cfd6-bc43-4d7f-a6a7-211ebc4e88e8 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:12:13.567600 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499158ac-a61a-4e54-9a46-e771dd5e6787 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:12:13.568423 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499158ac-a61a-4e54-9a46-e771dd5e6787 admin admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:12:13.568670 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499158ac-a61a-4e54-9a46-e771dd5e6787 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:12:13.568950 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499158ac-a61a-4e54-9a46-e771dd5e6787 admin admin] PATH_INFO: `/v3/domains` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:12:13.569610 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-499158ac-a61a-4e54-9a46-e771dd5e6787 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 11:12:13.569912 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-499158ac-a61a-4e54-9a46-e771dd5e6787 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:12:13.571630 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-499158ac-a61a-4e54-9a46-e771dd5e6787 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:12:13.587225 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1288/2558] 10.222.0.40 () {58 vars in 1153 bytes} [Fri Aug 26 11:12:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 11:12:13.592134 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df80275f-cde2-4f52-9bed-663c6edcff1b None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:12:13.601915 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df80275f-cde2-4f52-9bed-663c6edcff1b None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:12:13.606824 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83954d99-7608-4cce-8090-07b65f0af0a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:12:13.607691 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83954d99-7608-4cce-8090-07b65f0af0a3 admin admin] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:12:13.607968 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83954d99-7608-4cce-8090-07b65f0af0a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:12:13.608231 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83954d99-7608-4cce-8090-07b65f0af0a3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:12:13.608917 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83954d99-7608-4cce-8090-07b65f0af0a3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:12:13.610869 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83954d99-7608-4cce-8090-07b65f0af0a3 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:12:13.641204 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1271/2559] 10.222.0.40 () {60 vars in 1153 bytes} [Fri Aug 26 11:12:13 2022] POST /identity/v3/projects => generated 367 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 11:12:13.645636 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499158ac-a61a-4e54-9a46-e771dd5e6787 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:12:13.657031 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499158ac-a61a-4e54-9a46-e771dd5e6787 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:12:13.726328 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26bb9800-39dd-4de2-ac87-eacf82d6b09c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:12:13.726911 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bb9800-39dd-4de2-ac87-eacf82d6b09c admin admin] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:12:13.727122 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bb9800-39dd-4de2-ac87-eacf82d6b09c admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:12:13.727320 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bb9800-39dd-4de2-ac87-eacf82d6b09c admin admin] PATH_INFO: `/v3/users` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:12:13.727928 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26bb9800-39dd-4de2-ac87-eacf82d6b09c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:12:13.730015 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26bb9800-39dd-4de2-ac87-eacf82d6b09c admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:12:13.756781 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1289/2560] 10.222.0.40 () {60 vars in 1144 bytes} [Fri Aug 26 11:12:13 2022] POST /identity/v3/users => generated 284 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 11:12:13.761001 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83954d99-7608-4cce-8090-07b65f0af0a3 admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:12:13.772472 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83954d99-7608-4cce-8090-07b65f0af0a3 admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:12:13.777235 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c09bd2-5efd-4a55-aa26-f42bc7b21912 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:12:13.777856 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c09bd2-5efd-4a55-aa26-f42bc7b21912 admin admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:12:13.778202 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c09bd2-5efd-4a55-aa26-f42bc7b21912 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:12:13.778502 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c09bd2-5efd-4a55-aa26-f42bc7b21912 admin admin] PATH_INFO: `/v3/roles` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:12:13.779488 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c09bd2-5efd-4a55-aa26-f42bc7b21912 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:12:13.781074 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c09bd2-5efd-4a55-aa26-f42bc7b21912 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:12:13.792605 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1272/2561] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 26 11:12:13 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 11:12:13.797637 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26bb9800-39dd-4de2-ac87-eacf82d6b09c admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:12:13.808785 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26bb9800-39dd-4de2-ac87-eacf82d6b09c admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:12:13.814769 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4d3b67-9d8a-4a3d-9793-0072c375cc60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:12:13.815387 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4d3b67-9d8a-4a3d-9793-0072c375cc60 admin admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:12:13.815667 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4d3b67-9d8a-4a3d-9793-0072c375cc60 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:12:13.815938 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4d3b67-9d8a-4a3d-9793-0072c375cc60 admin admin] PATH_INFO: `/v3/roles` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:12:13.816588 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c4d3b67-9d8a-4a3d-9793-0072c375cc60 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:12:13.818215 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c4d3b67-9d8a-4a3d-9793-0072c375cc60 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:12:13.830789 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1290/2562] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 26 11:12:13 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 11:12:13.835383 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c09bd2-5efd-4a55-aa26-f42bc7b21912 admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:12:13.846235 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c09bd2-5efd-4a55-aa26-f42bc7b21912 admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:12:13.851298 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24defb10-884a-4654-b62d-4e2eaf2e1b0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:12:13.852014 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24defb10-884a-4654-b62d-4e2eaf2e1b0f admin admin] REQUEST_METHOD: `PUT` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:12:13.852290 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24defb10-884a-4654-b62d-4e2eaf2e1b0f admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:12:13.852597 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24defb10-884a-4654-b62d-4e2eaf2e1b0f admin admin] PATH_INFO: `/v3/projects/34d6ba90d14d40fa952d7efe5907e179/users/bdada9e11d324d158bd9ea1aac30211f/roles/23e174ce743b46eabe618c212691da55` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:12:13.873782 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24defb10-884a-4654-b62d-4e2eaf2e1b0f admin admin] RBAC: Authorizing `identity:create_grant(project_id=34d6ba90d14d40fa952d7efe5907e179, user_id=bdada9e11d324d158bd9ea1aac30211f, role_id=23e174ce743b46eabe618c212691da55)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:12:13.876546 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24defb10-884a-4654-b62d-4e2eaf2e1b0f admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:12:13.895987 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1273/2563] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 26 11:12:13 2022] PUT /identity/v3/projects/34d6ba90d14d40fa952d7efe5907e179/users/bdada9e11d324d158bd9ea1aac30211f/roles/23e174ce743b46eabe618c212691da55 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 11:12:13.911864 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c9b96b-8f61-4c2a-912c-ee32d72a12a4 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:12:13.912412 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c9b96b-8f61-4c2a-912c-ee32d72a12a4 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:12:13.912412 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c9b96b-8f61-4c2a-912c-ee32d72a12a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:12:13.932794 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 11:12:13.945215 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-62c9b96b-8f61-4c2a-912c-ee32d72a12a4 None None] MFA Rules not processed for user `bdada9e11d324d158bd9ea1aac30211f`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 11:12:13.946509 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-62c9b96b-8f61-4c2a-912c-ee32d72a12a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:12:14.017015 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1291/2564] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 11:12:13 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 11:12:14.028203 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24defb10-884a-4654-b62d-4e2eaf2e1b0f admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:12:14.030754 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-24defb10-884a-4654-b62d-4e2eaf2e1b0f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:12:14.094388 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24defb10-884a-4654-b62d-4e2eaf2e1b0f admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:12:14.098086 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee6dadd-172b-4f95-9d7f-c181ece11566 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c277338a7f814cc6b01c5419775031bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e30ccf0c1204727b786876c141dbcd2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:12:14.098685 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee6dadd-172b-4f95-9d7f-c181ece11566 service cinder] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:12:14.098957 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee6dadd-172b-4f95-9d7f-c181ece11566 service cinder] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:12:14.099222 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee6dadd-172b-4f95-9d7f-c181ece11566 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:12:14.108635 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee6dadd-172b-4f95-9d7f-c181ece11566 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:12:14.110088 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee6dadd-172b-4f95-9d7f-c181ece11566 service cinder] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:12:14.119301 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1274/2565] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 11:12:14 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 11:12:22.832583 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c9b96b-8f61-4c2a-912c-ee32d72a12a4 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:12:22.919629 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c9b96b-8f61-4c2a-912c-ee32d72a12a4 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:12:22.924813 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742098c2-c542-48ec-ac3d-d55c91ca209e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:12:22.925461 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742098c2-c542-48ec-ac3d-d55c91ca209e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:12:22.925741 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742098c2-c542-48ec-ac3d-d55c91ca209e admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:12:22.926022 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742098c2-c542-48ec-ac3d-d55c91ca209e admin admin] PATH_INFO: `/v3/users/bdada9e11d324d158bd9ea1aac30211f` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:12:22.927182 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-742098c2-c542-48ec-ac3d-d55c91ca209e admin admin] RBAC: Authorizing `identity:delete_user(user_id=bdada9e11d324d158bd9ea1aac30211f)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:12:22.929112 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-742098c2-c542-48ec-ac3d-d55c91ca209e admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:12:22.990454 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-742098c2-c542-48ec-ac3d-d55c91ca209e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bdada9e11d324d158bd9ea1aac30211f'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:12:22.991126 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-742098c2-c542-48ec-ac3d-d55c91ca209e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bdada9e11d324d158bd9ea1aac30211f'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:12:23.002571 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-742098c2-c542-48ec-ac3d-d55c91ca209e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bdada9e11d324d158bd9ea1aac30211f'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:12:23.003357 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-742098c2-c542-48ec-ac3d-d55c91ca209e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bdada9e11d324d158bd9ea1aac30211f'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:12:23.016927 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-742098c2-c542-48ec-ac3d-d55c91ca209e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bdada9e11d324d158bd9ea1aac30211f'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:12:23.036755 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1292/2566] 10.222.0.40 () {58 vars in 1224 bytes} [Fri Aug 26 11:12:22 2022] DELETE /identity/v3/users/bdada9e11d324d158bd9ea1aac30211f => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 11:12:23.041621 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ccdb369-06a6-4c4a-b900-9b2a807993bd None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:12:23.042128 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ccdb369-06a6-4c4a-b900-9b2a807993bd None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:12:23.042301 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ccdb369-06a6-4c4a-b900-9b2a807993bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:12:23.063865 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-8ccdb369-06a6-4c4a-b900-9b2a807993bd None None] MFA Rules not processed for user `ca155a48e2024c32af3fd2e19e5fac11`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 11:12:23.065243 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-8ccdb369-06a6-4c4a-b900-9b2a807993bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:12:23.157520 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1275/2567] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 11:12:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 11:12:23.166983 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742098c2-c542-48ec-ac3d-d55c91ca209e admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:12:23.168520 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-742098c2-c542-48ec-ac3d-d55c91ca209e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:12:23.194105 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 11:12:23.253938 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742098c2-c542-48ec-ac3d-d55c91ca209e admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:12:23.260399 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b720124-b2cf-40d5-bef0-faab36552053 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91da2acd2534423091537e1ba068d2b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e30ccf0c1204727b786876c141dbcd2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:12:23.261219 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b720124-b2cf-40d5-bef0-faab36552053 service neutron] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:12:23.261591 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b720124-b2cf-40d5-bef0-faab36552053 service neutron] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:12:23.262065 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b720124-b2cf-40d5-bef0-faab36552053 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:12:23.285872 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b720124-b2cf-40d5-bef0-faab36552053 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:12:23.287383 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b720124-b2cf-40d5-bef0-faab36552053 service neutron] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:12:23.298332 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1293/2568] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 11:12:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 11:12:23.590902 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ccdb369-06a6-4c4a-b900-9b2a807993bd None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:12:23.592879 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-8ccdb369-06a6-4c4a-b900-9b2a807993bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:12:23.605230 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ccdb369-06a6-4c4a-b900-9b2a807993bd None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:12:23.610813 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ffb06c-79f0-4b85-9dc8-0c51e7ceccd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:12:23.611462 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ffb06c-79f0-4b85-9dc8-0c51e7ceccd5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:12:23.611739 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ffb06c-79f0-4b85-9dc8-0c51e7ceccd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:12:23.612006 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ffb06c-79f0-4b85-9dc8-0c51e7ceccd5 admin admin] PATH_INFO: `/v3/projects/34d6ba90d14d40fa952d7efe5907e179` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:12:23.613155 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ffb06c-79f0-4b85-9dc8-0c51e7ceccd5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=34d6ba90d14d40fa952d7efe5907e179)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:12:23.615146 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ffb06c-79f0-4b85-9dc8-0c51e7ceccd5 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:12:23.633030 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=94,keystone:DELETE=3,keystone:INSERT=3 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 11:12:23.669077 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-32ffb06c-79f0-4b85-9dc8-0c51e7ceccd5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '34d6ba90d14d40fa952d7efe5907e179', 'request_id': 'req-32ffb06c-79f0-4b85-9dc8-0c51e7ceccd5'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:12:23.676409 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-32ffb06c-79f0-4b85-9dc8-0c51e7ceccd5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '34d6ba90d14d40fa952d7efe5907e179', 'request_id': 'req-32ffb06c-79f0-4b85-9dc8-0c51e7ceccd5'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:12:23.685364 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-32ffb06c-79f0-4b85-9dc8-0c51e7ceccd5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '34d6ba90d14d40fa952d7efe5907e179', 'request_id': 'req-32ffb06c-79f0-4b85-9dc8-0c51e7ceccd5'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:12:23.699523 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-32ffb06c-79f0-4b85-9dc8-0c51e7ceccd5 admin admin] The token cache is being invalidate because project 34d6ba90d14d40fa952d7efe5907e179 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81423) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 11:12:23.699877 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-32ffb06c-79f0-4b85-9dc8-0c51e7ceccd5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:12:23.700546 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-32ffb06c-79f0-4b85-9dc8-0c51e7ceccd5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:12:23.702296 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1276/2569] 10.222.0.40 () {58 vars in 1233 bytes} [Fri Aug 26 11:12:23 2022] DELETE /identity/v3/projects/34d6ba90d14d40fa952d7efe5907e179 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 11:12:23.707423 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f7e5b9-bfbd-427a-88d5-fbce1e1b9982 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:12:23.707699 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f7e5b9-bfbd-427a-88d5-fbce1e1b9982 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:12:23.707964 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f7e5b9-bfbd-427a-88d5-fbce1e1b9982 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:12:23.736365 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-66f7e5b9-bfbd-427a-88d5-fbce1e1b9982 None None] MFA Rules not processed for user `ca155a48e2024c32af3fd2e19e5fac11`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 11:12:23.738068 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-66f7e5b9-bfbd-427a-88d5-fbce1e1b9982 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:12:23.800010 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1294/2570] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 11:12:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 11:12:23.809360 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd6e098-314d-4764-85e0-8c2cf7af8def None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:12:23.809643 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd6e098-314d-4764-85e0-8c2cf7af8def None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:12:23.809912 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd6e098-314d-4764-85e0-8c2cf7af8def None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:12:23.833416 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-4fd6e098-314d-4764-85e0-8c2cf7af8def None None] MFA Rules not processed for user `ca155a48e2024c32af3fd2e19e5fac11`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 11:12:23.834690 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-4fd6e098-314d-4764-85e0-8c2cf7af8def None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:12:23.844096 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1277/2571] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 11:12:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 11:12:23.848633 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f7e5b9-bfbd-427a-88d5-fbce1e1b9982 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:12:23.859610 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f7e5b9-bfbd-427a-88d5-fbce1e1b9982 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:12:23.864531 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e62f980-2f6e-49d8-8311-12688ac89eea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:12:23.865164 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e62f980-2f6e-49d8-8311-12688ac89eea admin admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:12:23.865440 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e62f980-2f6e-49d8-8311-12688ac89eea admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:12:23.865708 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e62f980-2f6e-49d8-8311-12688ac89eea admin admin] PATH_INFO: `/v3/domains` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:12:23.866402 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e62f980-2f6e-49d8-8311-12688ac89eea admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 11:12:23.866702 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e62f980-2f6e-49d8-8311-12688ac89eea admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:12:23.868302 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e62f980-2f6e-49d8-8311-12688ac89eea admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:12:23.882397 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1295/2572] 10.222.0.40 () {58 vars in 1153 bytes} [Fri Aug 26 11:12:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 11:12:23.886907 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd6e098-314d-4764-85e0-8c2cf7af8def None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:12:23.897476 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd6e098-314d-4764-85e0-8c2cf7af8def None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:12:23.902133 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e393d43d-a10f-4a51-8a45-92163d88b2d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:12:23.902731 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e393d43d-a10f-4a51-8a45-92163d88b2d8 admin admin] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:12:23.903003 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e393d43d-a10f-4a51-8a45-92163d88b2d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:12:23.903268 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e393d43d-a10f-4a51-8a45-92163d88b2d8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:12:23.903886 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e393d43d-a10f-4a51-8a45-92163d88b2d8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:12:23.907266 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e393d43d-a10f-4a51-8a45-92163d88b2d8 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:12:23.936827 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1278/2573] 10.222.0.40 () {60 vars in 1153 bytes} [Fri Aug 26 11:12:23 2022] POST /identity/v3/projects => generated 377 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 26 11:12:23.941138 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e62f980-2f6e-49d8-8311-12688ac89eea admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:12:23.952338 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e62f980-2f6e-49d8-8311-12688ac89eea admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:12:24.012015 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7988154c-8d9b-4757-8a99-3db76163cce3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:12:24.012652 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7988154c-8d9b-4757-8a99-3db76163cce3 admin admin] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:12:24.012930 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7988154c-8d9b-4757-8a99-3db76163cce3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:12:24.013232 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7988154c-8d9b-4757-8a99-3db76163cce3 admin admin] PATH_INFO: `/v3/users` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:12:24.014506 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7988154c-8d9b-4757-8a99-3db76163cce3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:12:24.016453 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7988154c-8d9b-4757-8a99-3db76163cce3 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:12:24.041461 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1296/2574] 10.222.0.40 () {60 vars in 1144 bytes} [Fri Aug 26 11:12:23 2022] POST /identity/v3/users => generated 289 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 11:12:24.047990 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e393d43d-a10f-4a51-8a45-92163d88b2d8 admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:12:24.059755 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e393d43d-a10f-4a51-8a45-92163d88b2d8 admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:12:24.064982 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c5a174-bade-424a-9d03-bc05ba824176 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:12:24.065597 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c5a174-bade-424a-9d03-bc05ba824176 admin admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:12:24.065873 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c5a174-bade-424a-9d03-bc05ba824176 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:12:24.066173 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c5a174-bade-424a-9d03-bc05ba824176 admin admin] PATH_INFO: `/v3/roles` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:12:24.066825 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c5a174-bade-424a-9d03-bc05ba824176 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:12:24.068418 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c5a174-bade-424a-9d03-bc05ba824176 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:12:24.083726 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1279/2575] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 26 11:12:24 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 26 11:12:24.084907 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7988154c-8d9b-4757-8a99-3db76163cce3 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:12:24.096593 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7988154c-8d9b-4757-8a99-3db76163cce3 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:12:24.101185 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e532a763-7091-4ea5-b902-1c877f6dfe25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:12:24.101785 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e532a763-7091-4ea5-b902-1c877f6dfe25 admin admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:12:24.102086 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e532a763-7091-4ea5-b902-1c877f6dfe25 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:12:24.102361 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e532a763-7091-4ea5-b902-1c877f6dfe25 admin admin] PATH_INFO: `/v3/roles` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:12:24.102992 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e532a763-7091-4ea5-b902-1c877f6dfe25 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:12:24.104574 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e532a763-7091-4ea5-b902-1c877f6dfe25 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:12:24.115933 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1297/2576] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 26 11:12:24 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 26 11:12:24.120565 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c5a174-bade-424a-9d03-bc05ba824176 admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:12:24.132299 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c5a174-bade-424a-9d03-bc05ba824176 admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:12:24.139792 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e554d23-f213-4ded-9edc-e69f64b2d922 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:12:24.140515 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e554d23-f213-4ded-9edc-e69f64b2d922 admin admin] REQUEST_METHOD: `PUT` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:12:24.140791 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e554d23-f213-4ded-9edc-e69f64b2d922 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:12:24.141061 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e554d23-f213-4ded-9edc-e69f64b2d922 admin admin] PATH_INFO: `/v3/projects/43027bc071da4746a6178bcd91f3b2f0/users/a9f72666f7274b0a8b555b28093a390f/roles/23e174ce743b46eabe618c212691da55` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:12:24.156557 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e554d23-f213-4ded-9edc-e69f64b2d922 admin admin] RBAC: Authorizing `identity:create_grant(project_id=43027bc071da4746a6178bcd91f3b2f0, user_id=a9f72666f7274b0a8b555b28093a390f, role_id=23e174ce743b46eabe618c212691da55)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:12:24.159283 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e554d23-f213-4ded-9edc-e69f64b2d922 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:12:24.173364 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1280/2577] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 26 11:12:24 2022] PUT /identity/v3/projects/43027bc071da4746a6178bcd91f3b2f0/users/a9f72666f7274b0a8b555b28093a390f/roles/23e174ce743b46eabe618c212691da55 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 11:12:24.183627 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7fadb61-2794-4f24-9194-10b5bcb02b41 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:12:24.183907 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7fadb61-2794-4f24-9194-10b5bcb02b41 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:12:24.184178 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7fadb61-2794-4f24-9194-10b5bcb02b41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:12:24.217946 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-c7fadb61-2794-4f24-9194-10b5bcb02b41 None None] MFA Rules not processed for user `a9f72666f7274b0a8b555b28093a390f`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 11:12:24.219216 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-c7fadb61-2794-4f24-9194-10b5bcb02b41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:12:24.234999 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 11:12:24.306394 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1298/2578] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 11:12:24 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 11:12:24.311212 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e554d23-f213-4ded-9edc-e69f64b2d922 admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:12:24.386843 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e554d23-f213-4ded-9edc-e69f64b2d922 admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:12:24.392285 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d1f6ac-35a7-4538-aace-2c5e5fa182fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:12:24.392893 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d1f6ac-35a7-4538-aace-2c5e5fa182fa admin admin] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:12:24.393169 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d1f6ac-35a7-4538-aace-2c5e5fa182fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:12:24.393955 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d1f6ac-35a7-4538-aace-2c5e5fa182fa admin admin] PATH_INFO: `/v3/projects` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:12:24.394632 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1d1f6ac-35a7-4538-aace-2c5e5fa182fa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:12:24.396531 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1d1f6ac-35a7-4538-aace-2c5e5fa182fa admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:12:24.431693 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1281/2579] 10.222.0.40 () {60 vars in 1153 bytes} [Fri Aug 26 11:12:24 2022] POST /identity/v3/projects => generated 377 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 26 11:12:24.436515 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7fadb61-2794-4f24-9194-10b5bcb02b41 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:12:24.447734 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7fadb61-2794-4f24-9194-10b5bcb02b41 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:12:24.506972 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298221ea-7a5b-438b-a85a-908d5e6fbca2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:12:24.507593 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298221ea-7a5b-438b-a85a-908d5e6fbca2 admin admin] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:12:24.507867 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298221ea-7a5b-438b-a85a-908d5e6fbca2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:12:24.508135 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298221ea-7a5b-438b-a85a-908d5e6fbca2 admin admin] PATH_INFO: `/v3/users` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:12:24.508775 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-298221ea-7a5b-438b-a85a-908d5e6fbca2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:12:24.510739 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-298221ea-7a5b-438b-a85a-908d5e6fbca2 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:12:24.536300 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1299/2580] 10.222.0.40 () {60 vars in 1144 bytes} [Fri Aug 26 11:12:24 2022] POST /identity/v3/users => generated 295 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 11:12:24.541963 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d1f6ac-35a7-4538-aace-2c5e5fa182fa admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:12:24.562381 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d1f6ac-35a7-4538-aace-2c5e5fa182fa admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:12:24.567117 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d3752b-f38d-4b5c-9d46-c496bfa55c59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:12:24.567720 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d3752b-f38d-4b5c-9d46-c496bfa55c59 admin admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:12:24.567993 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d3752b-f38d-4b5c-9d46-c496bfa55c59 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:12:24.568257 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d3752b-f38d-4b5c-9d46-c496bfa55c59 admin admin] PATH_INFO: `/v3/roles` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:12:24.569119 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6d3752b-f38d-4b5c-9d46-c496bfa55c59 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:12:24.570743 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6d3752b-f38d-4b5c-9d46-c496bfa55c59 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:12:24.583030 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1282/2581] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 26 11:12:24 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 11:12:24.588128 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298221ea-7a5b-438b-a85a-908d5e6fbca2 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:12:24.599441 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298221ea-7a5b-438b-a85a-908d5e6fbca2 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:12:24.604275 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8757984-1fa7-433b-a4d0-e849163ee17f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:12:24.604945 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8757984-1fa7-433b-a4d0-e849163ee17f admin admin] REQUEST_METHOD: `PUT` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:12:24.605227 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8757984-1fa7-433b-a4d0-e849163ee17f admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:12:24.605505 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8757984-1fa7-433b-a4d0-e849163ee17f admin admin] PATH_INFO: `/v3/projects/56b534adfe9341e3b9affb09cce373a4/users/3fed6d779dad4a0785d57a4c72bc942f/roles/1f97e4602e2e4d6498913a81f9ae8d6a` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:12:24.626742 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8757984-1fa7-433b-a4d0-e849163ee17f admin admin] RBAC: Authorizing `identity:create_grant(project_id=56b534adfe9341e3b9affb09cce373a4, user_id=3fed6d779dad4a0785d57a4c72bc942f, role_id=1f97e4602e2e4d6498913a81f9ae8d6a)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:12:24.629526 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8757984-1fa7-433b-a4d0-e849163ee17f admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:12:24.646261 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1300/2582] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 26 11:12:24 2022] PUT /identity/v3/projects/56b534adfe9341e3b9affb09cce373a4/users/3fed6d779dad4a0785d57a4c72bc942f/roles/1f97e4602e2e4d6498913a81f9ae8d6a => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 11:12:24.651029 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d3752b-f38d-4b5c-9d46-c496bfa55c59 admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:12:24.664248 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d3752b-f38d-4b5c-9d46-c496bfa55c59 admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:12:24.674692 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 11:12:24.741647 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58c18a1-7e59-48d7-b074-368db13f8956 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:12:24.741647 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58c18a1-7e59-48d7-b074-368db13f8956 admin admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:12:24.741647 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58c18a1-7e59-48d7-b074-368db13f8956 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:12:24.741647 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58c18a1-7e59-48d7-b074-368db13f8956 admin admin] PATH_INFO: `/v3/roles` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:12:24.742209 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a58c18a1-7e59-48d7-b074-368db13f8956 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:12:24.743569 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a58c18a1-7e59-48d7-b074-368db13f8956 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:12:24.756193 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1283/2583] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 26 11:12:24 2022] GET /identity/v3/roles => generated 1420 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 11:12:24.760359 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8757984-1fa7-433b-a4d0-e849163ee17f admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:12:24.771629 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8757984-1fa7-433b-a4d0-e849163ee17f admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:12:24.776518 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b344ff0-b5c1-40be-8e7c-339a9c2c2473 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:12:24.777191 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b344ff0-b5c1-40be-8e7c-339a9c2c2473 admin admin] REQUEST_METHOD: `PUT` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:12:24.777468 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b344ff0-b5c1-40be-8e7c-339a9c2c2473 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:12:24.777742 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b344ff0-b5c1-40be-8e7c-339a9c2c2473 admin admin] PATH_INFO: `/v3/projects/56b534adfe9341e3b9affb09cce373a4/users/3fed6d779dad4a0785d57a4c72bc942f/roles/1f97e4602e2e4d6498913a81f9ae8d6a` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:12:24.780782 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b344ff0-b5c1-40be-8e7c-339a9c2c2473 admin admin] RBAC: Authorizing `identity:create_grant(project_id=56b534adfe9341e3b9affb09cce373a4, user_id=3fed6d779dad4a0785d57a4c72bc942f, role_id=1f97e4602e2e4d6498913a81f9ae8d6a)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:12:24.783489 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b344ff0-b5c1-40be-8e7c-339a9c2c2473 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:12:24.794679 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1301/2584] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 26 11:12:24 2022] PUT /identity/v3/projects/56b534adfe9341e3b9affb09cce373a4/users/3fed6d779dad4a0785d57a4c72bc942f/roles/1f97e4602e2e4d6498913a81f9ae8d6a => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 11:12:24.805379 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abc63e7-f92c-4b64-b222-a009476c1894 None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:12:24.805679 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abc63e7-f92c-4b64-b222-a009476c1894 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:12:24.805931 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abc63e7-f92c-4b64-b222-a009476c1894 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:12:24.838301 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-9abc63e7-f92c-4b64-b222-a009476c1894 None None] MFA Rules not processed for user `3fed6d779dad4a0785d57a4c72bc942f`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 11:12:24.839470 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-9abc63e7-f92c-4b64-b222-a009476c1894 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:12:24.923596 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1284/2585] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 11:12:24 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 11:12:24.934788 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b344ff0-b5c1-40be-8e7c-339a9c2c2473 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:12:24.937425 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-3b344ff0-b5c1-40be-8e7c-339a9c2c2473 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:12:25.000161 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b344ff0-b5c1-40be-8e7c-339a9c2c2473 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:12:25.005765 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef36c0b3-5977-43b5-9e2b-8aa20eb79f74 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c277338a7f814cc6b01c5419775031bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e30ccf0c1204727b786876c141dbcd2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:12:25.006417 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef36c0b3-5977-43b5-9e2b-8aa20eb79f74 service cinder] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:12:25.006696 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef36c0b3-5977-43b5-9e2b-8aa20eb79f74 service cinder] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:12:25.006964 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef36c0b3-5977-43b5-9e2b-8aa20eb79f74 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:12:25.030936 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 11:12:25.070934 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef36c0b3-5977-43b5-9e2b-8aa20eb79f74 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:12:25.072388 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef36c0b3-5977-43b5-9e2b-8aa20eb79f74 service cinder] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:12:25.083351 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1302/2586] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 11:12:24 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 11:12:30.869860 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abc63e7-f92c-4b64-b222-a009476c1894 None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:12:30.871343 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-9abc63e7-f92c-4b64-b222-a009476c1894 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:12:30.889893 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abc63e7-f92c-4b64-b222-a009476c1894 None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:12:30.893708 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e51b0d9-141d-4cab-a452-8c4d8345c65f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c277338a7f814cc6b01c5419775031bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e30ccf0c1204727b786876c141dbcd2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:12:30.894350 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e51b0d9-141d-4cab-a452-8c4d8345c65f service cinder] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:12:30.894682 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e51b0d9-141d-4cab-a452-8c4d8345c65f service cinder] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:12:30.894882 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e51b0d9-141d-4cab-a452-8c4d8345c65f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:12:30.908653 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e51b0d9-141d-4cab-a452-8c4d8345c65f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:12:30.910038 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e51b0d9-141d-4cab-a452-8c4d8345c65f service cinder] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:12:30.921030 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1285/2587] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 11:12:30 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 11:12:35.078550 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=17 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 11:12:35.260350 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef36c0b3-5977-43b5-9e2b-8aa20eb79f74 service cinder] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:12:35.261868 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-ef36c0b3-5977-43b5-9e2b-8aa20eb79f74 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:12:35.340690 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef36c0b3-5977-43b5-9e2b-8aa20eb79f74 service cinder] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:12:35.346439 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fabb448-e273-4d23-9277-1dbc4a400d07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:12:35.347169 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fabb448-e273-4d23-9277-1dbc4a400d07 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:12:35.347536 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fabb448-e273-4d23-9277-1dbc4a400d07 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:12:35.347702 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fabb448-e273-4d23-9277-1dbc4a400d07 admin admin] PATH_INFO: `/v3/users/0c023a6b12d24c138d104a2a4e8b02f5` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:12:35.348867 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fabb448-e273-4d23-9277-1dbc4a400d07 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0c023a6b12d24c138d104a2a4e8b02f5)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:12:35.350967 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fabb448-e273-4d23-9277-1dbc4a400d07 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:12:35.403944 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-3fabb448-e273-4d23-9277-1dbc4a400d07 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0c023a6b12d24c138d104a2a4e8b02f5'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:12:35.405173 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-3fabb448-e273-4d23-9277-1dbc4a400d07 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0c023a6b12d24c138d104a2a4e8b02f5'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:12:35.415774 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-3fabb448-e273-4d23-9277-1dbc4a400d07 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0c023a6b12d24c138d104a2a4e8b02f5'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:12:35.416498 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-3fabb448-e273-4d23-9277-1dbc4a400d07 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0c023a6b12d24c138d104a2a4e8b02f5'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:12:35.430571 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-3fabb448-e273-4d23-9277-1dbc4a400d07 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0c023a6b12d24c138d104a2a4e8b02f5'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:12:35.452506 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1303/2588] 10.222.0.40 () {58 vars in 1224 bytes} [Fri Aug 26 11:12:35 2022] DELETE /identity/v3/users/0c023a6b12d24c138d104a2a4e8b02f5 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 11:12:35.456610 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de92adf8-831f-4468-8702-410feeb7ce77 None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:12:35.457078 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de92adf8-831f-4468-8702-410feeb7ce77 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:12:35.458450 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de92adf8-831f-4468-8702-410feeb7ce77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:12:35.486369 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-de92adf8-831f-4468-8702-410feeb7ce77 None None] MFA Rules not processed for user `ca155a48e2024c32af3fd2e19e5fac11`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 11:12:35.487631 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-de92adf8-831f-4468-8702-410feeb7ce77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:12:35.583783 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1286/2589] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 11:12:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 11:12:35.595457 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fabb448-e273-4d23-9277-1dbc4a400d07 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:12:35.598212 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-3fabb448-e273-4d23-9277-1dbc4a400d07 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:12:35.690708 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fabb448-e273-4d23-9277-1dbc4a400d07 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:12:35.696279 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321cd9d2-af90-4985-b31a-690c19860eb2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91da2acd2534423091537e1ba068d2b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e30ccf0c1204727b786876c141dbcd2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:12:35.696894 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321cd9d2-af90-4985-b31a-690c19860eb2 service neutron] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:12:35.697171 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321cd9d2-af90-4985-b31a-690c19860eb2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:12:35.697463 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321cd9d2-af90-4985-b31a-690c19860eb2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:12:35.708691 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-321cd9d2-af90-4985-b31a-690c19860eb2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:12:35.710172 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-321cd9d2-af90-4985-b31a-690c19860eb2 service neutron] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:12:35.720934 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1304/2590] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 11:12:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 11:12:36.113816 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de92adf8-831f-4468-8702-410feeb7ce77 None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:12:36.116490 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-de92adf8-831f-4468-8702-410feeb7ce77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:12:36.144937 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de92adf8-831f-4468-8702-410feeb7ce77 None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:12:36.155080 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b725825e-b7d6-4648-b52d-8bb8d5ecd1c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:12:36.155785 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b725825e-b7d6-4648-b52d-8bb8d5ecd1c9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:12:36.156064 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b725825e-b7d6-4648-b52d-8bb8d5ecd1c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:12:36.156336 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b725825e-b7d6-4648-b52d-8bb8d5ecd1c9 admin admin] PATH_INFO: `/v3/projects/8ab915955b874605b29b06a9ac964778` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:12:36.157564 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b725825e-b7d6-4648-b52d-8bb8d5ecd1c9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8ab915955b874605b29b06a9ac964778)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:12:36.159678 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b725825e-b7d6-4648-b52d-8bb8d5ecd1c9 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:12:36.183416 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 11:12:36.246049 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-b725825e-b7d6-4648-b52d-8bb8d5ecd1c9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8ab915955b874605b29b06a9ac964778', 'request_id': 'req-b725825e-b7d6-4648-b52d-8bb8d5ecd1c9'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:12:36.254938 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-b725825e-b7d6-4648-b52d-8bb8d5ecd1c9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8ab915955b874605b29b06a9ac964778', 'request_id': 'req-b725825e-b7d6-4648-b52d-8bb8d5ecd1c9'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:12:36.263174 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-b725825e-b7d6-4648-b52d-8bb8d5ecd1c9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8ab915955b874605b29b06a9ac964778', 'request_id': 'req-b725825e-b7d6-4648-b52d-8bb8d5ecd1c9'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:12:36.281094 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-b725825e-b7d6-4648-b52d-8bb8d5ecd1c9 admin admin] The token cache is being invalidate because project 8ab915955b874605b29b06a9ac964778 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81423) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 11:12:36.281541 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-b725825e-b7d6-4648-b52d-8bb8d5ecd1c9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:12:36.282239 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-b725825e-b7d6-4648-b52d-8bb8d5ecd1c9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:12:36.283737 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1287/2591] 10.222.0.40 () {58 vars in 1233 bytes} [Fri Aug 26 11:12:36 2022] DELETE /identity/v3/projects/8ab915955b874605b29b06a9ac964778 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 11:12:36.288630 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf2bb0a-4c84-4118-95c7-2599cbda8b55 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:12:36.289152 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf2bb0a-4c84-4118-95c7-2599cbda8b55 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:12:36.289152 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf2bb0a-4c84-4118-95c7-2599cbda8b55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:12:36.313094 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-2bf2bb0a-4c84-4118-95c7-2599cbda8b55 None None] MFA Rules not processed for user `ca155a48e2024c32af3fd2e19e5fac11`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 11:12:36.314357 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-2bf2bb0a-4c84-4118-95c7-2599cbda8b55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:12:36.332752 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 11:12:36.385598 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1305/2592] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 11:12:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 11:12:36.393619 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71df1e2a-0069-46e2-8b6d-62a2c46fdf8c None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:12:36.393900 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71df1e2a-0069-46e2-8b6d-62a2c46fdf8c None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:12:36.394212 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71df1e2a-0069-46e2-8b6d-62a2c46fdf8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:12:36.418156 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-71df1e2a-0069-46e2-8b6d-62a2c46fdf8c None None] MFA Rules not processed for user `ca155a48e2024c32af3fd2e19e5fac11`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 11:12:36.419309 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-71df1e2a-0069-46e2-8b6d-62a2c46fdf8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:12:36.428533 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1288/2593] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 11:12:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 11:12:36.432947 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf2bb0a-4c84-4118-95c7-2599cbda8b55 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:12:36.443565 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf2bb0a-4c84-4118-95c7-2599cbda8b55 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:12:36.448875 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f045fa7-fc26-4e60-bf57-55c133642334 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:12:36.449511 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f045fa7-fc26-4e60-bf57-55c133642334 admin admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:12:36.449808 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f045fa7-fc26-4e60-bf57-55c133642334 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:12:36.450113 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f045fa7-fc26-4e60-bf57-55c133642334 admin admin] PATH_INFO: `/v3/domains` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:12:36.450777 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f045fa7-fc26-4e60-bf57-55c133642334 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 11:12:36.451137 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f045fa7-fc26-4e60-bf57-55c133642334 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:12:36.452837 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f045fa7-fc26-4e60-bf57-55c133642334 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:12:36.465847 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1306/2594] 10.222.0.40 () {58 vars in 1153 bytes} [Fri Aug 26 11:12:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 11:12:36.470839 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71df1e2a-0069-46e2-8b6d-62a2c46fdf8c None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:12:36.481118 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71df1e2a-0069-46e2-8b6d-62a2c46fdf8c None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:12:36.487140 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4411af53-8be1-4174-8c70-d5278f7058dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:12:36.487905 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4411af53-8be1-4174-8c70-d5278f7058dc admin admin] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:12:36.488191 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4411af53-8be1-4174-8c70-d5278f7058dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:12:36.488463 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4411af53-8be1-4174-8c70-d5278f7058dc admin admin] PATH_INFO: `/v3/projects` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:12:36.489115 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4411af53-8be1-4174-8c70-d5278f7058dc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:12:36.491069 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4411af53-8be1-4174-8c70-d5278f7058dc admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:12:36.554529 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1289/2595] 10.222.0.40 () {60 vars in 1153 bytes} [Fri Aug 26 11:12:36 2022] POST /identity/v3/projects => generated 383 bytes in 81 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 11:12:36.555946 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f045fa7-fc26-4e60-bf57-55c133642334 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:12:36.568033 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f045fa7-fc26-4e60-bf57-55c133642334 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:12:36.698385 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37f713a-9d27-45d9-ae90-c85cad1689b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:12:36.699032 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37f713a-9d27-45d9-ae90-c85cad1689b0 admin admin] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:12:36.699309 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37f713a-9d27-45d9-ae90-c85cad1689b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:12:36.699582 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37f713a-9d27-45d9-ae90-c85cad1689b0 admin admin] PATH_INFO: `/v3/users` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:12:36.700209 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d37f713a-9d27-45d9-ae90-c85cad1689b0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:12:36.702572 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d37f713a-9d27-45d9-ae90-c85cad1689b0 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:12:36.726032 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1307/2596] 10.222.0.40 () {60 vars in 1144 bytes} [Fri Aug 26 11:12:36 2022] POST /identity/v3/users => generated 292 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 11:12:36.730225 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4411af53-8be1-4174-8c70-d5278f7058dc admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:12:36.790772 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4411af53-8be1-4174-8c70-d5278f7058dc admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:12:36.796351 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c955e450-722a-474b-a495-0ce39437914b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:12:36.796979 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c955e450-722a-474b-a495-0ce39437914b admin admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:12:36.797255 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c955e450-722a-474b-a495-0ce39437914b admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:12:36.797523 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c955e450-722a-474b-a495-0ce39437914b admin admin] PATH_INFO: `/v3/roles` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:12:36.798202 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c955e450-722a-474b-a495-0ce39437914b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:12:36.799850 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c955e450-722a-474b-a495-0ce39437914b admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:12:36.813278 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1290/2597] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 26 11:12:36 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 11:12:36.817807 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37f713a-9d27-45d9-ae90-c85cad1689b0 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:12:36.829790 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37f713a-9d27-45d9-ae90-c85cad1689b0 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:12:36.836810 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b426c0-9891-46ad-9cbe-f26b638995e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:12:36.837534 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b426c0-9891-46ad-9cbe-f26b638995e5 admin admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:12:36.837883 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b426c0-9891-46ad-9cbe-f26b638995e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:12:36.838255 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b426c0-9891-46ad-9cbe-f26b638995e5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:12:36.838984 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b426c0-9891-46ad-9cbe-f26b638995e5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:12:36.840692 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b426c0-9891-46ad-9cbe-f26b638995e5 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:12:36.863639 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1308/2598] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 26 11:12:36 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 11:12:36.864727 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c955e450-722a-474b-a495-0ce39437914b admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:12:36.876099 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c955e450-722a-474b-a495-0ce39437914b admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:12:36.883563 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca87285-5e42-498b-96a0-c841c85638c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:12:36.884244 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca87285-5e42-498b-96a0-c841c85638c8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:12:36.884527 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca87285-5e42-498b-96a0-c841c85638c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:12:36.884802 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca87285-5e42-498b-96a0-c841c85638c8 admin admin] PATH_INFO: `/v3/projects/eb0b0a5e3578441aaed46d4660320fd1/users/c9d8be27d60649a6b61ceedc8370139b/roles/23e174ce743b46eabe618c212691da55` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:12:36.909632 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca87285-5e42-498b-96a0-c841c85638c8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=eb0b0a5e3578441aaed46d4660320fd1, user_id=c9d8be27d60649a6b61ceedc8370139b, role_id=23e174ce743b46eabe618c212691da55)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:12:36.912764 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca87285-5e42-498b-96a0-c841c85638c8 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:12:36.927883 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1291/2599] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 26 11:12:36 2022] PUT /identity/v3/projects/eb0b0a5e3578441aaed46d4660320fd1/users/c9d8be27d60649a6b61ceedc8370139b/roles/23e174ce743b46eabe618c212691da55 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 26 11:12:36.939315 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68108f51-0b6f-4b94-92b8-8d4fb56e3dc1 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:12:36.939599 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68108f51-0b6f-4b94-92b8-8d4fb56e3dc1 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:12:36.939874 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68108f51-0b6f-4b94-92b8-8d4fb56e3dc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:12:36.975074 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-68108f51-0b6f-4b94-92b8-8d4fb56e3dc1 None None] MFA Rules not processed for user `c9d8be27d60649a6b61ceedc8370139b`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 11:12:36.976649 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-68108f51-0b6f-4b94-92b8-8d4fb56e3dc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:12:37.056364 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 11:12:37.095836 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1309/2600] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 11:12:36 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 11:12:37.100436 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca87285-5e42-498b-96a0-c841c85638c8 admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:12:37.166012 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 11:12:37.188569 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca87285-5e42-498b-96a0-c841c85638c8 admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:12:37.197808 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609ef882-03fb-401a-815f-88782694b0ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:12:37.198633 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609ef882-03fb-401a-815f-88782694b0ee admin admin] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:12:37.198821 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609ef882-03fb-401a-815f-88782694b0ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:12:37.199095 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609ef882-03fb-401a-815f-88782694b0ee admin admin] PATH_INFO: `/v3/projects` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:12:37.199768 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-609ef882-03fb-401a-815f-88782694b0ee admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:12:37.203018 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-609ef882-03fb-401a-815f-88782694b0ee admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:12:37.241448 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1292/2601] 10.222.0.40 () {60 vars in 1153 bytes} [Fri Aug 26 11:12:37 2022] POST /identity/v3/projects => generated 385 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 26 11:12:37.247164 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68108f51-0b6f-4b94-92b8-8d4fb56e3dc1 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:12:37.279747 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68108f51-0b6f-4b94-92b8-8d4fb56e3dc1 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:12:37.360166 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f2483b-0ab2-4f6e-8112-7c6bf3b57590 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:12:37.361206 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f2483b-0ab2-4f6e-8112-7c6bf3b57590 admin admin] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:12:37.361700 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f2483b-0ab2-4f6e-8112-7c6bf3b57590 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:12:37.362582 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f2483b-0ab2-4f6e-8112-7c6bf3b57590 admin admin] PATH_INFO: `/v3/users` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:12:37.363743 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f2483b-0ab2-4f6e-8112-7c6bf3b57590 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:12:37.368023 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f2483b-0ab2-4f6e-8112-7c6bf3b57590 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:12:37.398157 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1310/2602] 10.222.0.40 () {60 vars in 1144 bytes} [Fri Aug 26 11:12:37 2022] POST /identity/v3/users => generated 293 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 26 11:12:37.402290 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609ef882-03fb-401a-815f-88782694b0ee admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:12:37.413425 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609ef882-03fb-401a-815f-88782694b0ee admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:12:37.418145 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc835df-892c-47b8-83f3-1c2e6a1381ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:12:37.418757 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc835df-892c-47b8-83f3-1c2e6a1381ef admin admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:12:37.419032 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc835df-892c-47b8-83f3-1c2e6a1381ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:12:37.419300 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc835df-892c-47b8-83f3-1c2e6a1381ef admin admin] PATH_INFO: `/v3/roles` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:12:37.420615 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc835df-892c-47b8-83f3-1c2e6a1381ef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:12:37.423854 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc835df-892c-47b8-83f3-1c2e6a1381ef admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:12:37.436491 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1293/2603] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 26 11:12:37 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 11:12:37.440751 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f2483b-0ab2-4f6e-8112-7c6bf3b57590 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:12:37.451617 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f2483b-0ab2-4f6e-8112-7c6bf3b57590 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:12:37.458243 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73cfb5c5-835a-4cb9-9ffa-c8ad9257a1d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:12:37.458945 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73cfb5c5-835a-4cb9-9ffa-c8ad9257a1d0 admin admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:12:37.459377 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73cfb5c5-835a-4cb9-9ffa-c8ad9257a1d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:12:37.459798 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73cfb5c5-835a-4cb9-9ffa-c8ad9257a1d0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:12:37.460952 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73cfb5c5-835a-4cb9-9ffa-c8ad9257a1d0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:12:37.463271 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73cfb5c5-835a-4cb9-9ffa-c8ad9257a1d0 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:12:37.475530 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1311/2604] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 26 11:12:37 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 11:12:37.480011 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc835df-892c-47b8-83f3-1c2e6a1381ef admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:12:37.491884 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc835df-892c-47b8-83f3-1c2e6a1381ef admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:12:37.496722 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f11511d-30f3-4fdc-a2cf-8b3ea2c5b511 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:12:37.497442 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f11511d-30f3-4fdc-a2cf-8b3ea2c5b511 admin admin] REQUEST_METHOD: `PUT` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:12:37.498044 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f11511d-30f3-4fdc-a2cf-8b3ea2c5b511 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:12:37.498811 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f11511d-30f3-4fdc-a2cf-8b3ea2c5b511 admin admin] PATH_INFO: `/v3/projects/2613388d971e4a4daeb66ed62ea82798/users/4b0eb15a799847a3b56a5c51d2a53ef0/roles/23e174ce743b46eabe618c212691da55` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:12:37.521421 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f11511d-30f3-4fdc-a2cf-8b3ea2c5b511 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2613388d971e4a4daeb66ed62ea82798, user_id=4b0eb15a799847a3b56a5c51d2a53ef0, role_id=23e174ce743b46eabe618c212691da55)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:12:37.521941 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f11511d-30f3-4fdc-a2cf-8b3ea2c5b511 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:12:37.537394 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1294/2605] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 26 11:12:37 2022] PUT /identity/v3/projects/2613388d971e4a4daeb66ed62ea82798/users/4b0eb15a799847a3b56a5c51d2a53ef0/roles/23e174ce743b46eabe618c212691da55 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 11:12:37.548700 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05dc8c8-6b02-47cc-9e46-23b0642f0724 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:12:37.549085 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05dc8c8-6b02-47cc-9e46-23b0642f0724 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:12:37.549447 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05dc8c8-6b02-47cc-9e46-23b0642f0724 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:12:37.582958 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-a05dc8c8-6b02-47cc-9e46-23b0642f0724 None None] MFA Rules not processed for user `4b0eb15a799847a3b56a5c51d2a53ef0`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 11:12:37.584246 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-a05dc8c8-6b02-47cc-9e46-23b0642f0724 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:12:37.660461 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1312/2606] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 11:12:37 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 11:12:37.665107 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f11511d-30f3-4fdc-a2cf-8b3ea2c5b511 admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:12:37.736795 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f11511d-30f3-4fdc-a2cf-8b3ea2c5b511 admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:12:37.743044 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb55479-532b-4fd0-ba76-db577260da4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:12:37.743661 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb55479-532b-4fd0-ba76-db577260da4b admin admin] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:12:37.744840 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb55479-532b-4fd0-ba76-db577260da4b admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:12:37.745200 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb55479-532b-4fd0-ba76-db577260da4b admin admin] PATH_INFO: `/v3/projects` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:12:37.746267 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb55479-532b-4fd0-ba76-db577260da4b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:12:37.748626 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb55479-532b-4fd0-ba76-db577260da4b admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:12:37.785440 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1295/2607] 10.222.0.40 () {60 vars in 1153 bytes} [Fri Aug 26 11:12:37 2022] POST /identity/v3/projects => generated 383 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 11:12:37.789822 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05dc8c8-6b02-47cc-9e46-23b0642f0724 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:12:37.802226 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05dc8c8-6b02-47cc-9e46-23b0642f0724 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:12:37.848689 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 11:12:37.871666 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56901705-a1a7-402b-8293-d6e64a07d8f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:12:37.872380 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56901705-a1a7-402b-8293-d6e64a07d8f3 admin admin] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:12:37.873109 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56901705-a1a7-402b-8293-d6e64a07d8f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:12:37.873357 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56901705-a1a7-402b-8293-d6e64a07d8f3 admin admin] PATH_INFO: `/v3/users` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:12:37.874097 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56901705-a1a7-402b-8293-d6e64a07d8f3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:12:37.876151 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56901705-a1a7-402b-8293-d6e64a07d8f3 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:12:37.905028 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1313/2608] 10.222.0.40 () {60 vars in 1144 bytes} [Fri Aug 26 11:12:37 2022] POST /identity/v3/users => generated 298 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 11:12:37.910326 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb55479-532b-4fd0-ba76-db577260da4b admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:12:37.925931 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb55479-532b-4fd0-ba76-db577260da4b admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:12:37.931335 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74bdc735-b62f-4873-9f1a-f3396a9aca7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:12:37.931953 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bdc735-b62f-4873-9f1a-f3396a9aca7c admin admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:12:37.932235 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bdc735-b62f-4873-9f1a-f3396a9aca7c admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:12:37.932507 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bdc735-b62f-4873-9f1a-f3396a9aca7c admin admin] PATH_INFO: `/v3/roles` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:12:37.933162 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74bdc735-b62f-4873-9f1a-f3396a9aca7c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:12:37.935489 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74bdc735-b62f-4873-9f1a-f3396a9aca7c admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:12:37.947559 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1296/2609] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 26 11:12:37 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 11:12:37.953021 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56901705-a1a7-402b-8293-d6e64a07d8f3 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:12:37.964942 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56901705-a1a7-402b-8293-d6e64a07d8f3 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:12:37.971443 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ea4bc4-eab5-4327-86d5-c4b338abb0e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:12:37.972218 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ea4bc4-eab5-4327-86d5-c4b338abb0e0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:12:37.972594 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ea4bc4-eab5-4327-86d5-c4b338abb0e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:12:37.972945 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ea4bc4-eab5-4327-86d5-c4b338abb0e0 admin admin] PATH_INFO: `/v3/projects/fd3e4e01d60f4963a710f699061dc591/users/eeb78fae918c49b08eaaabec27429aee/roles/1f97e4602e2e4d6498913a81f9ae8d6a` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:12:38.004434 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ea4bc4-eab5-4327-86d5-c4b338abb0e0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fd3e4e01d60f4963a710f699061dc591, user_id=eeb78fae918c49b08eaaabec27429aee, role_id=1f97e4602e2e4d6498913a81f9ae8d6a)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:12:38.007406 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ea4bc4-eab5-4327-86d5-c4b338abb0e0 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:12:38.024534 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1314/2610] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 26 11:12:37 2022] PUT /identity/v3/projects/fd3e4e01d60f4963a710f699061dc591/users/eeb78fae918c49b08eaaabec27429aee/roles/1f97e4602e2e4d6498913a81f9ae8d6a => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 11:12:38.028773 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74bdc735-b62f-4873-9f1a-f3396a9aca7c admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:12:38.043320 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74bdc735-b62f-4873-9f1a-f3396a9aca7c admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:12:38.116960 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153e55fd-2345-4843-88ef-004489cbc600 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:12:38.117581 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153e55fd-2345-4843-88ef-004489cbc600 admin admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:12:38.117858 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153e55fd-2345-4843-88ef-004489cbc600 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:12:38.118164 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153e55fd-2345-4843-88ef-004489cbc600 admin admin] PATH_INFO: `/v3/roles` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:12:38.119072 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-153e55fd-2345-4843-88ef-004489cbc600 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:12:38.120469 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-153e55fd-2345-4843-88ef-004489cbc600 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:12:38.147586 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1297/2611] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 26 11:12:38 2022] GET /identity/v3/roles => generated 1420 bytes in 116 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 11:12:38.149347 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ea4bc4-eab5-4327-86d5-c4b338abb0e0 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:12:38.164100 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ea4bc4-eab5-4327-86d5-c4b338abb0e0 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:12:38.168938 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74049998-e888-42b0-adcd-3a52499f99ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:12:38.169669 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74049998-e888-42b0-adcd-3a52499f99ba admin admin] REQUEST_METHOD: `PUT` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:12:38.169956 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74049998-e888-42b0-adcd-3a52499f99ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:12:38.170278 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74049998-e888-42b0-adcd-3a52499f99ba admin admin] PATH_INFO: `/v3/projects/fd3e4e01d60f4963a710f699061dc591/users/eeb78fae918c49b08eaaabec27429aee/roles/1f97e4602e2e4d6498913a81f9ae8d6a` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:12:38.172473 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74049998-e888-42b0-adcd-3a52499f99ba admin admin] RBAC: Authorizing `identity:create_grant(project_id=fd3e4e01d60f4963a710f699061dc591, user_id=eeb78fae918c49b08eaaabec27429aee, role_id=1f97e4602e2e4d6498913a81f9ae8d6a)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:12:38.175185 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74049998-e888-42b0-adcd-3a52499f99ba admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:12:38.186534 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1315/2612] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 26 11:12:38 2022] PUT /identity/v3/projects/fd3e4e01d60f4963a710f699061dc591/users/eeb78fae918c49b08eaaabec27429aee/roles/1f97e4602e2e4d6498913a81f9ae8d6a => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 26 11:12:38.197967 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa0f47d-4583-4e9d-9275-7fcb73670185 None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:12:38.198574 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa0f47d-4583-4e9d-9275-7fcb73670185 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:12:38.198859 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa0f47d-4583-4e9d-9275-7fcb73670185 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:12:38.220639 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 11:12:38.231833 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-0aa0f47d-4583-4e9d-9275-7fcb73670185 None None] MFA Rules not processed for user `eeb78fae918c49b08eaaabec27429aee`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 11:12:38.233246 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-0aa0f47d-4583-4e9d-9275-7fcb73670185 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:12:38.317299 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1298/2613] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 11:12:38 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 11:12:38.330401 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74049998-e888-42b0-adcd-3a52499f99ba admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:12:38.332372 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-74049998-e888-42b0-adcd-3a52499f99ba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:12:38.417900 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74049998-e888-42b0-adcd-3a52499f99ba admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:12:38.422036 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61afbc28-f212-4f0a-85d9-b41ac5d918eb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c277338a7f814cc6b01c5419775031bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e30ccf0c1204727b786876c141dbcd2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:12:38.422283 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61afbc28-f212-4f0a-85d9-b41ac5d918eb service cinder] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:12:38.422528 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61afbc28-f212-4f0a-85d9-b41ac5d918eb service cinder] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:12:38.422874 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61afbc28-f212-4f0a-85d9-b41ac5d918eb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:12:38.491428 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61afbc28-f212-4f0a-85d9-b41ac5d918eb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:12:38.492891 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61afbc28-f212-4f0a-85d9-b41ac5d918eb service cinder] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:12:38.503814 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1316/2614] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 11:12:38 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 174 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 11:12:39.406605 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73b393c-ac1e-4905-9cc3-dad953da7290 None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:12:39.407462 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73b393c-ac1e-4905-9cc3-dad953da7290 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:12:39.407462 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73b393c-ac1e-4905-9cc3-dad953da7290 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:12:39.426988 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-d73b393c-ac1e-4905-9cc3-dad953da7290 None None] MFA Rules not processed for user `85716f53e88048af9a8fdcbe4415979c`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 11:12:39.428215 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-d73b393c-ac1e-4905-9cc3-dad953da7290 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:12:39.485914 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1299/2615] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 11:12:39 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 11:12:41.137842 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61afbc28-f212-4f0a-85d9-b41ac5d918eb service cinder] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:12:41.148061 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61afbc28-f212-4f0a-85d9-b41ac5d918eb service cinder] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:12:41.153349 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb3acc9-888d-4dd6-ab97-2d69265f94e5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c277338a7f814cc6b01c5419775031bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e30ccf0c1204727b786876c141dbcd2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:12:41.154855 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb3acc9-888d-4dd6-ab97-2d69265f94e5 service cinder] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:12:41.155293 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb3acc9-888d-4dd6-ab97-2d69265f94e5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:12:41.155627 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb3acc9-888d-4dd6-ab97-2d69265f94e5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:12:41.207379 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 11:12:41.255015 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb3acc9-888d-4dd6-ab97-2d69265f94e5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:12:41.256384 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb3acc9-888d-4dd6-ab97-2d69265f94e5 service cinder] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:12:41.276396 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1317/2616] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 11:12:41 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 11:12:42.024720 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73b393c-ac1e-4905-9cc3-dad953da7290 None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:12:42.035872 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73b393c-ac1e-4905-9cc3-dad953da7290 None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:12:42.039903 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92308873-ada5-434d-a373-a19e0260a6bc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c277338a7f814cc6b01c5419775031bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e30ccf0c1204727b786876c141dbcd2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:12:42.040577 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92308873-ada5-434d-a373-a19e0260a6bc service cinder] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:12:42.040846 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92308873-ada5-434d-a373-a19e0260a6bc service cinder] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:12:42.041112 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92308873-ada5-434d-a373-a19e0260a6bc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:12:42.051245 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92308873-ada5-434d-a373-a19e0260a6bc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:12:42.052734 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92308873-ada5-434d-a373-a19e0260a6bc service cinder] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:12:42.063603 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1300/2617] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 11:12:42 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 11:12:48.134802 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb3acc9-888d-4dd6-ab97-2d69265f94e5 service cinder] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:12:48.207757 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb3acc9-888d-4dd6-ab97-2d69265f94e5 service cinder] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:12:48.213148 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3400bc78-57f8-4dd8-9726-d947c0c9fb34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:12:48.213882 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3400bc78-57f8-4dd8-9726-d947c0c9fb34 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:12:48.214197 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3400bc78-57f8-4dd8-9726-d947c0c9fb34 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:12:48.214471 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3400bc78-57f8-4dd8-9726-d947c0c9fb34 admin admin] PATH_INFO: `/v3/users/c9d8be27d60649a6b61ceedc8370139b` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:12:48.215773 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3400bc78-57f8-4dd8-9726-d947c0c9fb34 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c9d8be27d60649a6b61ceedc8370139b)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:12:48.218179 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3400bc78-57f8-4dd8-9726-d947c0c9fb34 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:12:48.269882 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-3400bc78-57f8-4dd8-9726-d947c0c9fb34 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c9d8be27d60649a6b61ceedc8370139b'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:12:48.270575 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-3400bc78-57f8-4dd8-9726-d947c0c9fb34 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c9d8be27d60649a6b61ceedc8370139b'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:12:48.277501 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-3400bc78-57f8-4dd8-9726-d947c0c9fb34 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c9d8be27d60649a6b61ceedc8370139b'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:12:48.278334 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-3400bc78-57f8-4dd8-9726-d947c0c9fb34 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c9d8be27d60649a6b61ceedc8370139b'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:12:48.292509 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-3400bc78-57f8-4dd8-9726-d947c0c9fb34 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c9d8be27d60649a6b61ceedc8370139b'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:12:48.308435 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1318/2618] 10.222.0.40 () {58 vars in 1224 bytes} [Fri Aug 26 11:12:48 2022] DELETE /identity/v3/users/c9d8be27d60649a6b61ceedc8370139b => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 11:12:48.312791 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51720c9-f45e-4196-ac07-7abf329c1f78 None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:12:48.313535 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51720c9-f45e-4196-ac07-7abf329c1f78 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:12:48.313535 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51720c9-f45e-4196-ac07-7abf329c1f78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:12:48.344077 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-a51720c9-f45e-4196-ac07-7abf329c1f78 None None] MFA Rules not processed for user `ca155a48e2024c32af3fd2e19e5fac11`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 11:12:48.345443 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-a51720c9-f45e-4196-ac07-7abf329c1f78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:12:48.425288 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1301/2619] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 11:12:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 11:12:48.436597 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3400bc78-57f8-4dd8-9726-d947c0c9fb34 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:12:48.438181 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-3400bc78-57f8-4dd8-9726-d947c0c9fb34 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:12:48.506365 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3400bc78-57f8-4dd8-9726-d947c0c9fb34 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:12:48.510114 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac2d2bb-0d87-48af-8f0b-fe79dd14f3a3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91da2acd2534423091537e1ba068d2b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e30ccf0c1204727b786876c141dbcd2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:12:48.510731 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac2d2bb-0d87-48af-8f0b-fe79dd14f3a3 service neutron] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:12:48.511011 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac2d2bb-0d87-48af-8f0b-fe79dd14f3a3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:12:48.511281 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac2d2bb-0d87-48af-8f0b-fe79dd14f3a3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:12:48.526026 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 11:12:48.529652 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac2d2bb-0d87-48af-8f0b-fe79dd14f3a3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:12:48.531519 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac2d2bb-0d87-48af-8f0b-fe79dd14f3a3 service neutron] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:12:48.543997 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1319/2620] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 11:12:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 11:12:48.833310 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51720c9-f45e-4196-ac07-7abf329c1f78 None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:12:48.835114 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-a51720c9-f45e-4196-ac07-7abf329c1f78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:12:48.848294 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51720c9-f45e-4196-ac07-7abf329c1f78 None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:12:48.853165 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92af0da9-eac2-49df-b34a-1c61ee97ba46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:12:48.853802 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92af0da9-eac2-49df-b34a-1c61ee97ba46 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:12:48.854110 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92af0da9-eac2-49df-b34a-1c61ee97ba46 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:12:48.854537 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92af0da9-eac2-49df-b34a-1c61ee97ba46 admin admin] PATH_INFO: `/v3/projects/eb0b0a5e3578441aaed46d4660320fd1` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:12:48.855724 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92af0da9-eac2-49df-b34a-1c61ee97ba46 admin admin] RBAC: Authorizing `identity:delete_project(project_id=eb0b0a5e3578441aaed46d4660320fd1)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:12:48.858307 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92af0da9-eac2-49df-b34a-1c61ee97ba46 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:12:48.882218 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 11:12:48.913672 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-92af0da9-eac2-49df-b34a-1c61ee97ba46 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eb0b0a5e3578441aaed46d4660320fd1', 'request_id': 'req-92af0da9-eac2-49df-b34a-1c61ee97ba46'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:12:48.919708 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-92af0da9-eac2-49df-b34a-1c61ee97ba46 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eb0b0a5e3578441aaed46d4660320fd1', 'request_id': 'req-92af0da9-eac2-49df-b34a-1c61ee97ba46'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:12:48.925607 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-92af0da9-eac2-49df-b34a-1c61ee97ba46 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eb0b0a5e3578441aaed46d4660320fd1', 'request_id': 'req-92af0da9-eac2-49df-b34a-1c61ee97ba46'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:12:48.938259 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-92af0da9-eac2-49df-b34a-1c61ee97ba46 admin admin] The token cache is being invalidate because project eb0b0a5e3578441aaed46d4660320fd1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81423) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 11:12:48.938539 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-92af0da9-eac2-49df-b34a-1c61ee97ba46 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:12:48.939147 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-92af0da9-eac2-49df-b34a-1c61ee97ba46 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:12:48.940612 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1302/2621] 10.222.0.40 () {58 vars in 1233 bytes} [Fri Aug 26 11:12:48 2022] DELETE /identity/v3/projects/eb0b0a5e3578441aaed46d4660320fd1 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 11:12:48.945699 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac2d2bb-0d87-48af-8f0b-fe79dd14f3a3 service neutron] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:12:48.957202 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac2d2bb-0d87-48af-8f0b-fe79dd14f3a3 service neutron] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:12:49.012546 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f2f105-a9f9-4973-b82e-34558ad4c8d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:12:49.013173 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f2f105-a9f9-4973-b82e-34558ad4c8d0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:12:49.014080 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f2f105-a9f9-4973-b82e-34558ad4c8d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:12:49.014414 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f2f105-a9f9-4973-b82e-34558ad4c8d0 admin admin] PATH_INFO: `/v3/users/4b0eb15a799847a3b56a5c51d2a53ef0` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:12:49.015730 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f2f105-a9f9-4973-b82e-34558ad4c8d0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4b0eb15a799847a3b56a5c51d2a53ef0)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:12:49.017650 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f2f105-a9f9-4973-b82e-34558ad4c8d0 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:12:49.072312 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-71f2f105-a9f9-4973-b82e-34558ad4c8d0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4b0eb15a799847a3b56a5c51d2a53ef0'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:12:49.073012 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-71f2f105-a9f9-4973-b82e-34558ad4c8d0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4b0eb15a799847a3b56a5c51d2a53ef0'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:12:49.080824 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-71f2f105-a9f9-4973-b82e-34558ad4c8d0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4b0eb15a799847a3b56a5c51d2a53ef0'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:12:49.081532 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-71f2f105-a9f9-4973-b82e-34558ad4c8d0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4b0eb15a799847a3b56a5c51d2a53ef0'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:12:49.094205 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-71f2f105-a9f9-4973-b82e-34558ad4c8d0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4b0eb15a799847a3b56a5c51d2a53ef0'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:12:49.109033 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1320/2622] 10.222.0.40 () {58 vars in 1224 bytes} [Fri Aug 26 11:12:48 2022] DELETE /identity/v3/users/4b0eb15a799847a3b56a5c51d2a53ef0 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 11:12:49.397268 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92af0da9-eac2-49df-b34a-1c61ee97ba46 admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:12:49.398778 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-92af0da9-eac2-49df-b34a-1c61ee97ba46 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:12:49.429566 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92af0da9-eac2-49df-b34a-1c61ee97ba46 admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:12:49.489261 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9310579c-a213-4543-978b-48c6122e7260 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:12:49.489798 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9310579c-a213-4543-978b-48c6122e7260 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:12:49.490101 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9310579c-a213-4543-978b-48c6122e7260 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:12:49.490375 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9310579c-a213-4543-978b-48c6122e7260 admin admin] PATH_INFO: `/v3/projects/2613388d971e4a4daeb66ed62ea82798` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:12:49.491496 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9310579c-a213-4543-978b-48c6122e7260 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2613388d971e4a4daeb66ed62ea82798)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:12:49.493468 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9310579c-a213-4543-978b-48c6122e7260 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:12:49.544085 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-9310579c-a213-4543-978b-48c6122e7260 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2613388d971e4a4daeb66ed62ea82798', 'request_id': 'req-9310579c-a213-4543-978b-48c6122e7260'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:12:49.551351 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-9310579c-a213-4543-978b-48c6122e7260 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2613388d971e4a4daeb66ed62ea82798', 'request_id': 'req-9310579c-a213-4543-978b-48c6122e7260'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:12:49.557029 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-9310579c-a213-4543-978b-48c6122e7260 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2613388d971e4a4daeb66ed62ea82798', 'request_id': 'req-9310579c-a213-4543-978b-48c6122e7260'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:12:49.571785 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-9310579c-a213-4543-978b-48c6122e7260 admin admin] The token cache is being invalidate because project 2613388d971e4a4daeb66ed62ea82798 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81423) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 11:12:49.572072 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-9310579c-a213-4543-978b-48c6122e7260 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:12:49.572889 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-9310579c-a213-4543-978b-48c6122e7260 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:12:49.574290 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1303/2623] 10.222.0.40 () {58 vars in 1233 bytes} [Fri Aug 26 11:12:49 2022] DELETE /identity/v3/projects/2613388d971e4a4daeb66ed62ea82798 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 11:12:49.578709 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f2f105-a9f9-4973-b82e-34558ad4c8d0 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:12:49.580115 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-71f2f105-a9f9-4973-b82e-34558ad4c8d0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:12:49.593515 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f2f105-a9f9-4973-b82e-34558ad4c8d0 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:12:49.664040 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462f165f-063c-4489-8381-7cabc806cc30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:12:49.664708 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462f165f-063c-4489-8381-7cabc806cc30 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:12:49.665221 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462f165f-063c-4489-8381-7cabc806cc30 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:12:49.665221 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462f165f-063c-4489-8381-7cabc806cc30 admin admin] PATH_INFO: `/v3/users/eeb78fae918c49b08eaaabec27429aee` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:12:49.666688 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-462f165f-063c-4489-8381-7cabc806cc30 admin admin] RBAC: Authorizing `identity:delete_user(user_id=eeb78fae918c49b08eaaabec27429aee)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:12:49.669239 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-462f165f-063c-4489-8381-7cabc806cc30 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:12:49.696733 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 11:12:49.736989 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-462f165f-063c-4489-8381-7cabc806cc30 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eeb78fae918c49b08eaaabec27429aee'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:12:49.737694 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-462f165f-063c-4489-8381-7cabc806cc30 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eeb78fae918c49b08eaaabec27429aee'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:12:49.745448 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-462f165f-063c-4489-8381-7cabc806cc30 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eeb78fae918c49b08eaaabec27429aee'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:12:49.746177 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-462f165f-063c-4489-8381-7cabc806cc30 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eeb78fae918c49b08eaaabec27429aee'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:12:49.759484 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-462f165f-063c-4489-8381-7cabc806cc30 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eeb78fae918c49b08eaaabec27429aee'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:12:49.774034 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1321/2624] 10.222.0.40 () {58 vars in 1224 bytes} [Fri Aug 26 11:12:49 2022] DELETE /identity/v3/users/eeb78fae918c49b08eaaabec27429aee => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 11:12:50.083906 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9310579c-a213-4543-978b-48c6122e7260 admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:12:50.085418 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-9310579c-a213-4543-978b-48c6122e7260 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:12:50.102630 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9310579c-a213-4543-978b-48c6122e7260 admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:12:50.136923 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 11:12:50.167164 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca0b850-1dbc-4dce-b296-009c3feb5600 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:12:50.167914 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca0b850-1dbc-4dce-b296-009c3feb5600 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:12:50.168227 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca0b850-1dbc-4dce-b296-009c3feb5600 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:12:50.168503 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca0b850-1dbc-4dce-b296-009c3feb5600 admin admin] PATH_INFO: `/v3/projects/fd3e4e01d60f4963a710f699061dc591` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:12:50.170240 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca0b850-1dbc-4dce-b296-009c3feb5600 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fd3e4e01d60f4963a710f699061dc591)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:12:50.172154 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca0b850-1dbc-4dce-b296-009c3feb5600 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:12:50.224391 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-2ca0b850-1dbc-4dce-b296-009c3feb5600 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fd3e4e01d60f4963a710f699061dc591', 'request_id': 'req-2ca0b850-1dbc-4dce-b296-009c3feb5600'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:12:50.230961 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-2ca0b850-1dbc-4dce-b296-009c3feb5600 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fd3e4e01d60f4963a710f699061dc591', 'request_id': 'req-2ca0b850-1dbc-4dce-b296-009c3feb5600'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:12:50.240912 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-2ca0b850-1dbc-4dce-b296-009c3feb5600 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fd3e4e01d60f4963a710f699061dc591', 'request_id': 'req-2ca0b850-1dbc-4dce-b296-009c3feb5600'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:12:50.254667 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-2ca0b850-1dbc-4dce-b296-009c3feb5600 admin admin] The token cache is being invalidate because project fd3e4e01d60f4963a710f699061dc591 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81423) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 11:12:50.254874 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-2ca0b850-1dbc-4dce-b296-009c3feb5600 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:12:50.255494 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-2ca0b850-1dbc-4dce-b296-009c3feb5600 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:12:50.257156 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1304/2625] 10.222.0.40 () {58 vars in 1233 bytes} [Fri Aug 26 11:12:50 2022] DELETE /identity/v3/projects/fd3e4e01d60f4963a710f699061dc591 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 11:12:50.262290 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29d4edc-9c2c-48d7-b703-3837e2f62697 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:12:50.262569 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29d4edc-9c2c-48d7-b703-3837e2f62697 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:12:50.262839 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29d4edc-9c2c-48d7-b703-3837e2f62697 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:12:50.315630 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-d29d4edc-9c2c-48d7-b703-3837e2f62697 None None] MFA Rules not processed for user `ca155a48e2024c32af3fd2e19e5fac11`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 11:12:50.316847 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-d29d4edc-9c2c-48d7-b703-3837e2f62697 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:12:50.377931 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1322/2626] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 11:12:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 11:12:50.385474 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc81f62e-d714-4fcf-9ee1-1e28589b46a7 None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:12:50.385756 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc81f62e-d714-4fcf-9ee1-1e28589b46a7 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:12:50.386065 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc81f62e-d714-4fcf-9ee1-1e28589b46a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:12:50.406525 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-dc81f62e-d714-4fcf-9ee1-1e28589b46a7 None None] MFA Rules not processed for user `ca155a48e2024c32af3fd2e19e5fac11`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 11:12:50.407735 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-dc81f62e-d714-4fcf-9ee1-1e28589b46a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:12:50.415819 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1305/2627] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 11:12:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 11:12:50.421003 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29d4edc-9c2c-48d7-b703-3837e2f62697 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:12:50.430882 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29d4edc-9c2c-48d7-b703-3837e2f62697 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:12:50.435692 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6444dbab-15fa-4077-a02a-79f3208a0fcc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:12:50.436494 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6444dbab-15fa-4077-a02a-79f3208a0fcc admin admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:12:50.436609 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6444dbab-15fa-4077-a02a-79f3208a0fcc admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:12:50.436884 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6444dbab-15fa-4077-a02a-79f3208a0fcc admin admin] PATH_INFO: `/v3/domains` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:12:50.437466 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6444dbab-15fa-4077-a02a-79f3208a0fcc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 11:12:50.437769 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6444dbab-15fa-4077-a02a-79f3208a0fcc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:12:50.439424 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6444dbab-15fa-4077-a02a-79f3208a0fcc admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:12:50.457449 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1323/2628] 10.222.0.40 () {58 vars in 1153 bytes} [Fri Aug 26 11:12:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 11:12:50.459063 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc81f62e-d714-4fcf-9ee1-1e28589b46a7 None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:12:50.469434 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc81f62e-d714-4fcf-9ee1-1e28589b46a7 None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:12:50.474250 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f23f8d7-dd65-4deb-b6cb-b96bcd3bc319 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:12:50.474851 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f23f8d7-dd65-4deb-b6cb-b96bcd3bc319 admin admin] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:12:50.475126 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f23f8d7-dd65-4deb-b6cb-b96bcd3bc319 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:12:50.475392 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f23f8d7-dd65-4deb-b6cb-b96bcd3bc319 admin admin] PATH_INFO: `/v3/projects` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:12:50.476010 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f23f8d7-dd65-4deb-b6cb-b96bcd3bc319 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:12:50.477886 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f23f8d7-dd65-4deb-b6cb-b96bcd3bc319 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:12:50.510155 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1306/2629] 10.222.0.40 () {60 vars in 1153 bytes} [Fri Aug 26 11:12:50 2022] POST /identity/v3/projects => generated 373 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 11:12:50.511284 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6444dbab-15fa-4077-a02a-79f3208a0fcc admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:12:50.523920 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6444dbab-15fa-4077-a02a-79f3208a0fcc admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:12:50.585059 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:DELETE=9,keystone:SELECT=90,keystone:INSERT=1 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 11:12:50.594300 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3980c09-ca14-4918-b134-e811f005ee9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:12:50.594963 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3980c09-ca14-4918-b134-e811f005ee9f admin admin] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:12:50.595392 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3980c09-ca14-4918-b134-e811f005ee9f admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:12:50.595685 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3980c09-ca14-4918-b134-e811f005ee9f admin admin] PATH_INFO: `/v3/users` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:12:50.596800 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3980c09-ca14-4918-b134-e811f005ee9f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:12:50.598908 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3980c09-ca14-4918-b134-e811f005ee9f admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:12:50.622595 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1324/2630] 10.222.0.40 () {60 vars in 1144 bytes} [Fri Aug 26 11:12:50 2022] POST /identity/v3/users => generated 287 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 26 11:12:50.627079 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f23f8d7-dd65-4deb-b6cb-b96bcd3bc319 admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:12:50.638871 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f23f8d7-dd65-4deb-b6cb-b96bcd3bc319 admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:12:50.643699 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344a9f13-8667-4940-8adb-3f02e7ac3172 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:12:50.644313 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344a9f13-8667-4940-8adb-3f02e7ac3172 admin admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:12:50.644592 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344a9f13-8667-4940-8adb-3f02e7ac3172 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:12:50.644861 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344a9f13-8667-4940-8adb-3f02e7ac3172 admin admin] PATH_INFO: `/v3/roles` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:12:50.645509 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-344a9f13-8667-4940-8adb-3f02e7ac3172 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:12:50.647124 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-344a9f13-8667-4940-8adb-3f02e7ac3172 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:12:50.658396 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1307/2631] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 26 11:12:50 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 11:12:50.662363 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3980c09-ca14-4918-b134-e811f005ee9f admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:12:50.674589 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3980c09-ca14-4918-b134-e811f005ee9f admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:12:50.679583 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbdc3ed9-9dd4-4f20-9619-946f81abd0f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:12:50.680203 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbdc3ed9-9dd4-4f20-9619-946f81abd0f3 admin admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:12:50.680482 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbdc3ed9-9dd4-4f20-9619-946f81abd0f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:12:50.680749 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbdc3ed9-9dd4-4f20-9619-946f81abd0f3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:12:50.681405 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbdc3ed9-9dd4-4f20-9619-946f81abd0f3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:12:50.683032 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbdc3ed9-9dd4-4f20-9619-946f81abd0f3 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:12:50.695122 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1325/2632] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 26 11:12:50 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 11:12:50.699736 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344a9f13-8667-4940-8adb-3f02e7ac3172 admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:12:50.711346 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344a9f13-8667-4940-8adb-3f02e7ac3172 admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:12:50.716781 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2676dfa2-d56a-415c-958d-67e6f98b3559 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:12:50.717597 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2676dfa2-d56a-415c-958d-67e6f98b3559 admin admin] REQUEST_METHOD: `PUT` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:12:50.717914 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2676dfa2-d56a-415c-958d-67e6f98b3559 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:12:50.718190 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2676dfa2-d56a-415c-958d-67e6f98b3559 admin admin] PATH_INFO: `/v3/projects/ef07a23929104d298d85485c33a0411e/users/b4ea31e78ceb46f8bc92d86211da5072/roles/23e174ce743b46eabe618c212691da55` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:12:50.735673 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2676dfa2-d56a-415c-958d-67e6f98b3559 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ef07a23929104d298d85485c33a0411e, user_id=b4ea31e78ceb46f8bc92d86211da5072, role_id=23e174ce743b46eabe618c212691da55)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:12:50.738423 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2676dfa2-d56a-415c-958d-67e6f98b3559 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:12:50.753025 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1308/2633] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 26 11:12:50 2022] PUT /identity/v3/projects/ef07a23929104d298d85485c33a0411e/users/b4ea31e78ceb46f8bc92d86211da5072/roles/23e174ce743b46eabe618c212691da55 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 11:12:50.758281 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eeb9927-8fc5-4027-be3e-8d82e7f8d91e None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:12:50.758501 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eeb9927-8fc5-4027-be3e-8d82e7f8d91e None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:12:50.758780 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eeb9927-8fc5-4027-be3e-8d82e7f8d91e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:12:50.780087 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-3eeb9927-8fc5-4027-be3e-8d82e7f8d91e None None] MFA Rules not processed for user `ca155a48e2024c32af3fd2e19e5fac11`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 11:12:50.781367 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-3eeb9927-8fc5-4027-be3e-8d82e7f8d91e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:12:50.846049 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1326/2634] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 11:12:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 11:12:50.858518 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2676dfa2-d56a-415c-958d-67e6f98b3559 admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:12:50.860484 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-2676dfa2-d56a-415c-958d-67e6f98b3559 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:12:50.938917 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2676dfa2-d56a-415c-958d-67e6f98b3559 admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:12:50.943008 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816340d3-5107-441b-b7a9-6b6aa3697446 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91da2acd2534423091537e1ba068d2b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e30ccf0c1204727b786876c141dbcd2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:12:50.943629 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816340d3-5107-441b-b7a9-6b6aa3697446 service neutron] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:12:50.943904 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816340d3-5107-441b-b7a9-6b6aa3697446 service neutron] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:12:50.944169 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816340d3-5107-441b-b7a9-6b6aa3697446 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:12:50.954160 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-816340d3-5107-441b-b7a9-6b6aa3697446 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:12:50.955612 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-816340d3-5107-441b-b7a9-6b6aa3697446 service neutron] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:12:50.967004 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1309/2635] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 11:12:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 26 11:12:51.540557 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eeb9927-8fc5-4027-be3e-8d82e7f8d91e None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:12:51.542223 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-3eeb9927-8fc5-4027-be3e-8d82e7f8d91e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:12:51.653806 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eeb9927-8fc5-4027-be3e-8d82e7f8d91e None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:12:51.661122 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df17cf02-64f3-42a3-8677-efd06d003bde service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f29c5dfed31f42a9b5cbc0cd72e9fdc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e30ccf0c1204727b786876c141dbcd2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:12:51.661858 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df17cf02-64f3-42a3-8677-efd06d003bde service placement] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:12:51.662320 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df17cf02-64f3-42a3-8677-efd06d003bde service placement] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:12:51.662734 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df17cf02-64f3-42a3-8677-efd06d003bde service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:12:51.664991 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-df17cf02-64f3-42a3-8677-efd06d003bde service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:12:51.678775 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df17cf02-64f3-42a3-8677-efd06d003bde service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:12:51.680445 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df17cf02-64f3-42a3-8677-efd06d003bde service placement] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:12:51.697970 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1327/2636] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 11:12:51 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 11:12:51.715256 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a36c954-a07c-419a-a934-2b3d42c742f9 None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:12:51.715816 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a36c954-a07c-419a-a934-2b3d42c742f9 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:12:51.716041 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a36c954-a07c-419a-a934-2b3d42c742f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:12:51.724675 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 11:12:51.755953 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-8a36c954-a07c-419a-a934-2b3d42c742f9 None None] MFA Rules not processed for user `b4ea31e78ceb46f8bc92d86211da5072`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 11:12:51.757193 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-8a36c954-a07c-419a-a934-2b3d42c742f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:12:51.840501 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1310/2637] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 11:12:51 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 11:12:51.852201 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df17cf02-64f3-42a3-8677-efd06d003bde service placement] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:12:51.854072 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-df17cf02-64f3-42a3-8677-efd06d003bde service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:12:51.924705 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 11:12:51.934162 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df17cf02-64f3-42a3-8677-efd06d003bde service placement] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:12:51.940377 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789e1ee2-53b7-4c3f-8b83-ba6784ab9f4d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c277338a7f814cc6b01c5419775031bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e30ccf0c1204727b786876c141dbcd2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:12:51.941329 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789e1ee2-53b7-4c3f-8b83-ba6784ab9f4d service cinder] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:12:51.941571 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789e1ee2-53b7-4c3f-8b83-ba6784ab9f4d service cinder] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:12:51.941886 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789e1ee2-53b7-4c3f-8b83-ba6784ab9f4d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:12:51.954610 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-789e1ee2-53b7-4c3f-8b83-ba6784ab9f4d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:12:51.956481 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-789e1ee2-53b7-4c3f-8b83-ba6784ab9f4d service cinder] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:12:51.970799 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1328/2638] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 11:12:51 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 11:12:53.456152 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e804b8d-aa67-4553-8ef2-00f601ae57ba None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:12:53.456688 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e804b8d-aa67-4553-8ef2-00f601ae57ba None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:12:53.456688 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e804b8d-aa67-4553-8ef2-00f601ae57ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:12:53.477579 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-5e804b8d-aa67-4553-8ef2-00f601ae57ba None None] MFA Rules not processed for user `b4ea31e78ceb46f8bc92d86211da5072`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 11:12:53.478918 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-5e804b8d-aa67-4553-8ef2-00f601ae57ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:12:53.487498 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1311/2639] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 11:12:53 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 11:12:56.472593 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789e1ee2-53b7-4c3f-8b83-ba6784ab9f4d service cinder] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:12:56.474747 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-789e1ee2-53b7-4c3f-8b83-ba6784ab9f4d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:12:56.490668 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789e1ee2-53b7-4c3f-8b83-ba6784ab9f4d service cinder] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:12:56.495555 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15267bf9-8a48-439c-86f4-824c2215ad57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:12:56.496338 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15267bf9-8a48-439c-86f4-824c2215ad57 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:12:56.496544 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15267bf9-8a48-439c-86f4-824c2215ad57 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:12:56.496823 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15267bf9-8a48-439c-86f4-824c2215ad57 admin admin] PATH_INFO: `/v3/users/a9f72666f7274b0a8b555b28093a390f` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:12:56.498846 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15267bf9-8a48-439c-86f4-824c2215ad57 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a9f72666f7274b0a8b555b28093a390f)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:12:56.500807 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15267bf9-8a48-439c-86f4-824c2215ad57 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:12:56.555742 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-15267bf9-8a48-439c-86f4-824c2215ad57 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a9f72666f7274b0a8b555b28093a390f'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:12:56.556316 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-15267bf9-8a48-439c-86f4-824c2215ad57 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a9f72666f7274b0a8b555b28093a390f'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:12:56.564710 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-15267bf9-8a48-439c-86f4-824c2215ad57 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a9f72666f7274b0a8b555b28093a390f'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:12:56.565498 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-15267bf9-8a48-439c-86f4-824c2215ad57 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a9f72666f7274b0a8b555b28093a390f'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:12:56.578301 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-15267bf9-8a48-439c-86f4-824c2215ad57 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a9f72666f7274b0a8b555b28093a390f'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:12:56.600422 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1329/2640] 10.222.0.40 () {58 vars in 1224 bytes} [Fri Aug 26 11:12:56 2022] DELETE /identity/v3/users/a9f72666f7274b0a8b555b28093a390f => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 11:12:56.606128 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3207fe75-2435-4545-8383-63284a3c5d2a None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:12:56.606645 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3207fe75-2435-4545-8383-63284a3c5d2a None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:12:56.606779 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3207fe75-2435-4545-8383-63284a3c5d2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:12:56.628300 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-3207fe75-2435-4545-8383-63284a3c5d2a None None] MFA Rules not processed for user `ca155a48e2024c32af3fd2e19e5fac11`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 11:12:56.629756 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-3207fe75-2435-4545-8383-63284a3c5d2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:12:56.693271 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1312/2641] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 11:12:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 11:12:56.704693 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15267bf9-8a48-439c-86f4-824c2215ad57 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:12:56.706564 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-15267bf9-8a48-439c-86f4-824c2215ad57 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:12:56.801049 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15267bf9-8a48-439c-86f4-824c2215ad57 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:12:56.805172 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a953e663-4dba-4dd6-991a-41c202f0ed3f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91da2acd2534423091537e1ba068d2b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e30ccf0c1204727b786876c141dbcd2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:12:56.806095 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a953e663-4dba-4dd6-991a-41c202f0ed3f service neutron] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:12:56.806095 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a953e663-4dba-4dd6-991a-41c202f0ed3f service neutron] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:12:56.806361 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a953e663-4dba-4dd6-991a-41c202f0ed3f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:12:56.817002 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a953e663-4dba-4dd6-991a-41c202f0ed3f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:12:56.818527 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a953e663-4dba-4dd6-991a-41c202f0ed3f service neutron] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:12:56.829746 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1330/2642] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 11:12:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 11:12:57.150577 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3207fe75-2435-4545-8383-63284a3c5d2a None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:12:57.152069 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-3207fe75-2435-4545-8383-63284a3c5d2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:12:57.170252 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3207fe75-2435-4545-8383-63284a3c5d2a None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:12:57.176161 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28167c5f-bcb8-4e70-a6ea-9215ce1168de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:12:57.176811 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28167c5f-bcb8-4e70-a6ea-9215ce1168de admin admin] REQUEST_METHOD: `DELETE` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:12:57.177256 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28167c5f-bcb8-4e70-a6ea-9215ce1168de admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:12:57.180457 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28167c5f-bcb8-4e70-a6ea-9215ce1168de admin admin] PATH_INFO: `/v3/projects/43027bc071da4746a6178bcd91f3b2f0` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:12:57.183460 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28167c5f-bcb8-4e70-a6ea-9215ce1168de admin admin] RBAC: Authorizing `identity:delete_project(project_id=43027bc071da4746a6178bcd91f3b2f0)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:12:57.186273 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28167c5f-bcb8-4e70-a6ea-9215ce1168de admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:12:57.240772 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-28167c5f-bcb8-4e70-a6ea-9215ce1168de admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '43027bc071da4746a6178bcd91f3b2f0', 'request_id': 'req-28167c5f-bcb8-4e70-a6ea-9215ce1168de'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:12:57.251055 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-28167c5f-bcb8-4e70-a6ea-9215ce1168de admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '43027bc071da4746a6178bcd91f3b2f0', 'request_id': 'req-28167c5f-bcb8-4e70-a6ea-9215ce1168de'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:12:57.257371 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-28167c5f-bcb8-4e70-a6ea-9215ce1168de admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '43027bc071da4746a6178bcd91f3b2f0', 'request_id': 'req-28167c5f-bcb8-4e70-a6ea-9215ce1168de'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:12:57.273071 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-28167c5f-bcb8-4e70-a6ea-9215ce1168de admin admin] The token cache is being invalidate because project 43027bc071da4746a6178bcd91f3b2f0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81423) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 11:12:57.273811 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-28167c5f-bcb8-4e70-a6ea-9215ce1168de admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:12:57.274371 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-28167c5f-bcb8-4e70-a6ea-9215ce1168de admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:12:57.277173 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1313/2643] 10.222.0.40 () {58 vars in 1233 bytes} [Fri Aug 26 11:12:57 2022] DELETE /identity/v3/projects/43027bc071da4746a6178bcd91f3b2f0 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 11:12:57.281427 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a953e663-4dba-4dd6-991a-41c202f0ed3f service neutron] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:12:57.291439 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a953e663-4dba-4dd6-991a-41c202f0ed3f service neutron] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:12:57.358904 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 11:12:57.392198 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ab6ed8-c517-4fff-90a8-3bf5581bf47f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:12:57.392941 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ab6ed8-c517-4fff-90a8-3bf5581bf47f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:12:57.393820 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ab6ed8-c517-4fff-90a8-3bf5581bf47f admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:12:57.393820 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ab6ed8-c517-4fff-90a8-3bf5581bf47f admin admin] PATH_INFO: `/v3/users/3fed6d779dad4a0785d57a4c72bc942f` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:12:57.394784 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ab6ed8-c517-4fff-90a8-3bf5581bf47f admin admin] RBAC: Authorizing `identity:delete_user(user_id=3fed6d779dad4a0785d57a4c72bc942f)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:12:57.397767 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ab6ed8-c517-4fff-90a8-3bf5581bf47f admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:12:57.460532 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-44ab6ed8-c517-4fff-90a8-3bf5581bf47f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3fed6d779dad4a0785d57a4c72bc942f'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:12:57.461191 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-44ab6ed8-c517-4fff-90a8-3bf5581bf47f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3fed6d779dad4a0785d57a4c72bc942f'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:12:57.469112 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-44ab6ed8-c517-4fff-90a8-3bf5581bf47f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3fed6d779dad4a0785d57a4c72bc942f'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:12:57.469796 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-44ab6ed8-c517-4fff-90a8-3bf5581bf47f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3fed6d779dad4a0785d57a4c72bc942f'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:12:57.483360 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-44ab6ed8-c517-4fff-90a8-3bf5581bf47f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3fed6d779dad4a0785d57a4c72bc942f'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:12:57.498847 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1331/2644] 10.222.0.40 () {58 vars in 1224 bytes} [Fri Aug 26 11:12:57 2022] DELETE /identity/v3/users/3fed6d779dad4a0785d57a4c72bc942f => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 11:12:58.290408 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28167c5f-bcb8-4e70-a6ea-9215ce1168de admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:12:58.293284 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-28167c5f-bcb8-4e70-a6ea-9215ce1168de admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:12:58.317429 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28167c5f-bcb8-4e70-a6ea-9215ce1168de admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:12:58.324624 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 11:12:58.401551 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fddd5a7d-8ce9-4c75-b639-5ce5ecfb1f14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:12:58.402367 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fddd5a7d-8ce9-4c75-b639-5ce5ecfb1f14 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:12:58.403271 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fddd5a7d-8ce9-4c75-b639-5ce5ecfb1f14 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:12:58.403271 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fddd5a7d-8ce9-4c75-b639-5ce5ecfb1f14 admin admin] PATH_INFO: `/v3/projects/56b534adfe9341e3b9affb09cce373a4` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:12:58.404460 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fddd5a7d-8ce9-4c75-b639-5ce5ecfb1f14 admin admin] RBAC: Authorizing `identity:delete_project(project_id=56b534adfe9341e3b9affb09cce373a4)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:12:58.407301 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fddd5a7d-8ce9-4c75-b639-5ce5ecfb1f14 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:12:58.481637 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-fddd5a7d-8ce9-4c75-b639-5ce5ecfb1f14 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '56b534adfe9341e3b9affb09cce373a4', 'request_id': 'req-fddd5a7d-8ce9-4c75-b639-5ce5ecfb1f14'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:12:58.489749 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-fddd5a7d-8ce9-4c75-b639-5ce5ecfb1f14 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '56b534adfe9341e3b9affb09cce373a4', 'request_id': 'req-fddd5a7d-8ce9-4c75-b639-5ce5ecfb1f14'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:12:58.496929 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-fddd5a7d-8ce9-4c75-b639-5ce5ecfb1f14 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '56b534adfe9341e3b9affb09cce373a4', 'request_id': 'req-fddd5a7d-8ce9-4c75-b639-5ce5ecfb1f14'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:12:58.516712 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-fddd5a7d-8ce9-4c75-b639-5ce5ecfb1f14 admin admin] The token cache is being invalidate because project 56b534adfe9341e3b9affb09cce373a4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81423) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 11:12:58.517007 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-fddd5a7d-8ce9-4c75-b639-5ce5ecfb1f14 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:12:58.517736 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-fddd5a7d-8ce9-4c75-b639-5ce5ecfb1f14 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:12:58.519476 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1314/2645] 10.222.0.40 () {58 vars in 1233 bytes} [Fri Aug 26 11:12:58 2022] DELETE /identity/v3/projects/56b534adfe9341e3b9affb09cce373a4 => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 11:12:58.525225 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7a1272-c042-4001-af45-62277ab0b2ef None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:12:58.526265 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7a1272-c042-4001-af45-62277ab0b2ef None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:12:58.526771 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7a1272-c042-4001-af45-62277ab0b2ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:12:58.549689 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-fd7a1272-c042-4001-af45-62277ab0b2ef None None] MFA Rules not processed for user `ca155a48e2024c32af3fd2e19e5fac11`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 11:12:58.551086 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-fd7a1272-c042-4001-af45-62277ab0b2ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:12:58.615304 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1332/2646] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 11:12:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 11:12:58.625553 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee79f8f5-d9b0-4629-a139-a2733cfb237b None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:12:58.625836 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee79f8f5-d9b0-4629-a139-a2733cfb237b None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:12:58.626137 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee79f8f5-d9b0-4629-a139-a2733cfb237b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:12:58.647551 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-ee79f8f5-d9b0-4629-a139-a2733cfb237b None None] MFA Rules not processed for user `ca155a48e2024c32af3fd2e19e5fac11`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 11:12:58.648790 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-ee79f8f5-d9b0-4629-a139-a2733cfb237b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:12:58.658228 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1315/2647] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 11:12:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 11:12:58.662450 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7a1272-c042-4001-af45-62277ab0b2ef None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:12:58.674128 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7a1272-c042-4001-af45-62277ab0b2ef None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:12:58.679503 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06ae577-e84b-4ae8-a0cc-9d8f320a7f46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:12:58.680201 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06ae577-e84b-4ae8-a0cc-9d8f320a7f46 admin admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:12:58.680549 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06ae577-e84b-4ae8-a0cc-9d8f320a7f46 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:12:58.680890 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06ae577-e84b-4ae8-a0cc-9d8f320a7f46 admin admin] PATH_INFO: `/v3/domains` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:12:58.681782 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e06ae577-e84b-4ae8-a0cc-9d8f320a7f46 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 11:12:58.682190 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e06ae577-e84b-4ae8-a0cc-9d8f320a7f46 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:12:58.684079 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e06ae577-e84b-4ae8-a0cc-9d8f320a7f46 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:12:58.702981 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1333/2648] 10.222.0.40 () {58 vars in 1153 bytes} [Fri Aug 26 11:12:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 11:12:58.704159 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee79f8f5-d9b0-4629-a139-a2733cfb237b None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:12:58.716958 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee79f8f5-d9b0-4629-a139-a2733cfb237b None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:12:58.722277 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28bf411-e87e-4f51-845e-2dcbbbbf95e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:12:58.722965 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28bf411-e87e-4f51-845e-2dcbbbbf95e8 admin admin] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:12:58.723530 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28bf411-e87e-4f51-845e-2dcbbbbf95e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:12:58.723671 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28bf411-e87e-4f51-845e-2dcbbbbf95e8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:12:58.724337 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a28bf411-e87e-4f51-845e-2dcbbbbf95e8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:12:58.726349 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a28bf411-e87e-4f51-845e-2dcbbbbf95e8 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:12:58.761374 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1316/2649] 10.222.0.40 () {60 vars in 1153 bytes} [Fri Aug 26 11:12:58 2022] POST /identity/v3/projects => generated 383 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 26 11:12:58.765895 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06ae577-e84b-4ae8-a0cc-9d8f320a7f46 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:12:58.776666 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06ae577-e84b-4ae8-a0cc-9d8f320a7f46 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:12:58.827461 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 11:12:58.838436 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea51a6a-cb00-420a-968b-de591fa43b9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:12:58.839283 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea51a6a-cb00-420a-968b-de591fa43b9f admin admin] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:12:58.839904 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea51a6a-cb00-420a-968b-de591fa43b9f admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:12:58.840508 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea51a6a-cb00-420a-968b-de591fa43b9f admin admin] PATH_INFO: `/v3/users` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:12:58.842437 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea51a6a-cb00-420a-968b-de591fa43b9f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:12:58.844291 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea51a6a-cb00-420a-968b-de591fa43b9f admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:12:58.871305 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1334/2650] 10.222.0.40 () {60 vars in 1144 bytes} [Fri Aug 26 11:12:58 2022] POST /identity/v3/users => generated 292 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 11:12:58.875822 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28bf411-e87e-4f51-845e-2dcbbbbf95e8 admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:12:58.887100 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28bf411-e87e-4f51-845e-2dcbbbbf95e8 admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:12:58.892646 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ee5d3d-2e46-407a-98b8-b3b4678f7189 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:12:58.893259 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ee5d3d-2e46-407a-98b8-b3b4678f7189 admin admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:12:58.893533 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ee5d3d-2e46-407a-98b8-b3b4678f7189 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:12:58.893797 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ee5d3d-2e46-407a-98b8-b3b4678f7189 admin admin] PATH_INFO: `/v3/roles` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:12:58.894496 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ee5d3d-2e46-407a-98b8-b3b4678f7189 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:12:58.896087 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ee5d3d-2e46-407a-98b8-b3b4678f7189 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:12:58.909054 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1317/2651] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 26 11:12:58 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 11:12:58.912994 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea51a6a-cb00-420a-968b-de591fa43b9f admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:12:58.924609 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea51a6a-cb00-420a-968b-de591fa43b9f admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:12:58.929572 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24aedf3b-d694-44e5-80e2-d2436a30a19b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:12:58.930603 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24aedf3b-d694-44e5-80e2-d2436a30a19b admin admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:12:58.930603 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24aedf3b-d694-44e5-80e2-d2436a30a19b admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:12:58.930877 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24aedf3b-d694-44e5-80e2-d2436a30a19b admin admin] PATH_INFO: `/v3/roles` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:12:58.931391 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24aedf3b-d694-44e5-80e2-d2436a30a19b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:12:58.933151 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24aedf3b-d694-44e5-80e2-d2436a30a19b admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:12:58.944659 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1335/2652] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 26 11:12:58 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 11:12:58.948879 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ee5d3d-2e46-407a-98b8-b3b4678f7189 admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:12:58.960189 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ee5d3d-2e46-407a-98b8-b3b4678f7189 admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:12:58.965491 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f0b5cd-a302-4a54-8560-a191469515ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:12:58.967566 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f0b5cd-a302-4a54-8560-a191469515ce admin admin] REQUEST_METHOD: `PUT` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:12:58.967811 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f0b5cd-a302-4a54-8560-a191469515ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:12:58.968091 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f0b5cd-a302-4a54-8560-a191469515ce admin admin] PATH_INFO: `/v3/projects/76613fc75a8a4e0796a561ede228cd33/users/cf9bc46faca14a4885b5514c6e44d890/roles/23e174ce743b46eabe618c212691da55` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:12:58.983934 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f0b5cd-a302-4a54-8560-a191469515ce admin admin] RBAC: Authorizing `identity:create_grant(project_id=76613fc75a8a4e0796a561ede228cd33, user_id=cf9bc46faca14a4885b5514c6e44d890, role_id=23e174ce743b46eabe618c212691da55)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:12:58.986635 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f0b5cd-a302-4a54-8560-a191469515ce admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:12:59.001543 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1318/2653] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 26 11:12:58 2022] PUT /identity/v3/projects/76613fc75a8a4e0796a561ede228cd33/users/cf9bc46faca14a4885b5514c6e44d890/roles/23e174ce743b46eabe618c212691da55 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 11:12:59.020865 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4caf3e07-fd8a-4838-ab95-f55b67142dc8 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:12:59.020992 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4caf3e07-fd8a-4838-ab95-f55b67142dc8 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:12:59.021404 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4caf3e07-fd8a-4838-ab95-f55b67142dc8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:12:59.053817 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-4caf3e07-fd8a-4838-ab95-f55b67142dc8 None None] MFA Rules not processed for user `cf9bc46faca14a4885b5514c6e44d890`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 11:12:59.055082 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-4caf3e07-fd8a-4838-ab95-f55b67142dc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:12:59.150484 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1336/2654] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 11:12:59 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 11:12:59.162795 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f0b5cd-a302-4a54-8560-a191469515ce admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:12:59.164960 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-f3f0b5cd-a302-4a54-8560-a191469515ce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:12:59.223521 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 11:12:59.244615 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f0b5cd-a302-4a54-8560-a191469515ce admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:12:59.249372 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68f6253-86a7-40ac-907c-9e30eb9f81ef service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c277338a7f814cc6b01c5419775031bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e30ccf0c1204727b786876c141dbcd2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:12:59.250005 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68f6253-86a7-40ac-907c-9e30eb9f81ef service cinder] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:12:59.250308 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68f6253-86a7-40ac-907c-9e30eb9f81ef service cinder] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:12:59.250573 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68f6253-86a7-40ac-907c-9e30eb9f81ef service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:12:59.259656 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e68f6253-86a7-40ac-907c-9e30eb9f81ef service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:12:59.261107 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e68f6253-86a7-40ac-907c-9e30eb9f81ef service cinder] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:12:59.272114 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1319/2655] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 11:12:59 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 11:13:05.411102 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea6d5d5-a688-4357-9678-ff77c60e6cf7 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:13:05.411645 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea6d5d5-a688-4357-9678-ff77c60e6cf7 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:13:05.411645 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea6d5d5-a688-4357-9678-ff77c60e6cf7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:13:05.433253 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-bea6d5d5-a688-4357-9678-ff77c60e6cf7 None None] MFA Rules not processed for user `9e8355dc324e49d881b9f0fc5c2923ad`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 11:13:05.434510 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-bea6d5d5-a688-4357-9678-ff77c60e6cf7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:13:05.495628 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1337/2656] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 11:13:05 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 11:13:06.944037 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68f6253-86a7-40ac-907c-9e30eb9f81ef service cinder] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:13:07.033346 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68f6253-86a7-40ac-907c-9e30eb9f81ef service cinder] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:13:07.037960 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63f2bf6-2965-46e8-8da7-7c76194d0b84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:13:07.038832 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63f2bf6-2965-46e8-8da7-7c76194d0b84 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:13:07.038954 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63f2bf6-2965-46e8-8da7-7c76194d0b84 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:13:07.039205 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63f2bf6-2965-46e8-8da7-7c76194d0b84 admin admin] PATH_INFO: `/v3/users/cf9bc46faca14a4885b5514c6e44d890` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:13:07.040364 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e63f2bf6-2965-46e8-8da7-7c76194d0b84 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cf9bc46faca14a4885b5514c6e44d890)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:13:07.042320 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e63f2bf6-2965-46e8-8da7-7c76194d0b84 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:13:07.106372 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-e63f2bf6-2965-46e8-8da7-7c76194d0b84 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cf9bc46faca14a4885b5514c6e44d890'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:13:07.114647 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-e63f2bf6-2965-46e8-8da7-7c76194d0b84 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cf9bc46faca14a4885b5514c6e44d890'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:13:07.115264 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-e63f2bf6-2965-46e8-8da7-7c76194d0b84 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cf9bc46faca14a4885b5514c6e44d890'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:13:07.128815 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-e63f2bf6-2965-46e8-8da7-7c76194d0b84 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cf9bc46faca14a4885b5514c6e44d890'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:13:07.129546 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-e63f2bf6-2965-46e8-8da7-7c76194d0b84 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cf9bc46faca14a4885b5514c6e44d890'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:13:07.144918 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1320/2657] 10.222.0.40 () {58 vars in 1224 bytes} [Fri Aug 26 11:13:06 2022] DELETE /identity/v3/users/cf9bc46faca14a4885b5514c6e44d890 => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 11:13:07.150277 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5872dc5-2f65-46cf-a662-83103520056e None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:13:07.151291 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5872dc5-2f65-46cf-a662-83103520056e None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:13:07.151723 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5872dc5-2f65-46cf-a662-83103520056e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:13:07.179807 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-a5872dc5-2f65-46cf-a662-83103520056e None None] MFA Rules not processed for user `ca155a48e2024c32af3fd2e19e5fac11`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 11:13:07.181023 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-a5872dc5-2f65-46cf-a662-83103520056e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:13:07.259846 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1338/2658] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 11:13:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 11:13:07.269743 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63f2bf6-2965-46e8-8da7-7c76194d0b84 admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:13:07.271229 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-e63f2bf6-2965-46e8-8da7-7c76194d0b84 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:13:07.335822 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63f2bf6-2965-46e8-8da7-7c76194d0b84 admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:13:07.339502 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55cbdf63-1497-4125-af34-9290ef73d677 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91da2acd2534423091537e1ba068d2b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e30ccf0c1204727b786876c141dbcd2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:13:07.340102 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55cbdf63-1497-4125-af34-9290ef73d677 service neutron] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:13:07.340402 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55cbdf63-1497-4125-af34-9290ef73d677 service neutron] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:13:07.340666 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55cbdf63-1497-4125-af34-9290ef73d677 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:13:07.351032 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55cbdf63-1497-4125-af34-9290ef73d677 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:13:07.352584 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55cbdf63-1497-4125-af34-9290ef73d677 service neutron] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:13:07.358864 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 11:13:07.364538 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1321/2659] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 11:13:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 11:13:07.793760 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5872dc5-2f65-46cf-a662-83103520056e None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:13:07.795787 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-a5872dc5-2f65-46cf-a662-83103520056e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:13:07.809699 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 11:13:07.817961 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5872dc5-2f65-46cf-a662-83103520056e None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:13:07.832631 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c3b1ff-a409-4b1d-85ee-cf12aa9e57f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:13:07.833658 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c3b1ff-a409-4b1d-85ee-cf12aa9e57f2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:13:07.834098 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c3b1ff-a409-4b1d-85ee-cf12aa9e57f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:13:07.834761 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c3b1ff-a409-4b1d-85ee-cf12aa9e57f2 admin admin] PATH_INFO: `/v3/projects/76613fc75a8a4e0796a561ede228cd33` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:13:07.836717 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c3b1ff-a409-4b1d-85ee-cf12aa9e57f2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=76613fc75a8a4e0796a561ede228cd33)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:13:07.840364 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c3b1ff-a409-4b1d-85ee-cf12aa9e57f2 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:13:07.922206 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-b6c3b1ff-a409-4b1d-85ee-cf12aa9e57f2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '76613fc75a8a4e0796a561ede228cd33', 'request_id': 'req-b6c3b1ff-a409-4b1d-85ee-cf12aa9e57f2'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:13:07.935335 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-b6c3b1ff-a409-4b1d-85ee-cf12aa9e57f2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '76613fc75a8a4e0796a561ede228cd33', 'request_id': 'req-b6c3b1ff-a409-4b1d-85ee-cf12aa9e57f2'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:13:07.948562 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-b6c3b1ff-a409-4b1d-85ee-cf12aa9e57f2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '76613fc75a8a4e0796a561ede228cd33', 'request_id': 'req-b6c3b1ff-a409-4b1d-85ee-cf12aa9e57f2'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:13:07.975532 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-b6c3b1ff-a409-4b1d-85ee-cf12aa9e57f2 admin admin] The token cache is being invalidate because project 76613fc75a8a4e0796a561ede228cd33 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81424) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 11:13:07.976154 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-b6c3b1ff-a409-4b1d-85ee-cf12aa9e57f2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:13:07.976844 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-b6c3b1ff-a409-4b1d-85ee-cf12aa9e57f2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:13:07.978661 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1339/2660] 10.222.0.40 () {58 vars in 1233 bytes} [Fri Aug 26 11:13:07 2022] DELETE /identity/v3/projects/76613fc75a8a4e0796a561ede228cd33 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 11:13:07.983609 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3748c007-94cf-415c-86fa-190337c5f726 None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:13:07.984175 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3748c007-94cf-415c-86fa-190337c5f726 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:13:07.984175 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3748c007-94cf-415c-86fa-190337c5f726 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:13:08.005698 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-3748c007-94cf-415c-86fa-190337c5f726 None None] MFA Rules not processed for user `ca155a48e2024c32af3fd2e19e5fac11`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 11:13:08.007028 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-3748c007-94cf-415c-86fa-190337c5f726 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:13:08.097573 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1322/2661] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 11:13:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 11:13:08.106566 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d92ad2e-8342-457f-aba4-e0d60ddfd4b8 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:13:08.106928 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d92ad2e-8342-457f-aba4-e0d60ddfd4b8 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:13:08.107389 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d92ad2e-8342-457f-aba4-e0d60ddfd4b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:13:08.131147 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-7d92ad2e-8342-457f-aba4-e0d60ddfd4b8 None None] MFA Rules not processed for user `ca155a48e2024c32af3fd2e19e5fac11`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 11:13:08.132578 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-7d92ad2e-8342-457f-aba4-e0d60ddfd4b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:13:08.142333 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1340/2662] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 11:13:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 11:13:08.147619 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3748c007-94cf-415c-86fa-190337c5f726 None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:13:08.162047 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3748c007-94cf-415c-86fa-190337c5f726 None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:13:08.168761 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6b6405-7625-4b91-8203-63bc12f53a3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:13:08.169601 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6b6405-7625-4b91-8203-63bc12f53a3e admin admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:13:08.170367 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6b6405-7625-4b91-8203-63bc12f53a3e admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:13:08.170367 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6b6405-7625-4b91-8203-63bc12f53a3e admin admin] PATH_INFO: `/v3/domains` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:13:08.170835 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb6b6405-7625-4b91-8203-63bc12f53a3e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 11:13:08.171139 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb6b6405-7625-4b91-8203-63bc12f53a3e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:13:08.172756 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb6b6405-7625-4b91-8203-63bc12f53a3e admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:13:08.187658 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1323/2663] 10.222.0.40 () {58 vars in 1153 bytes} [Fri Aug 26 11:13:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 11:13:08.191945 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d92ad2e-8342-457f-aba4-e0d60ddfd4b8 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:13:08.202566 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d92ad2e-8342-457f-aba4-e0d60ddfd4b8 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:13:08.207752 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4e4c79-bbea-4779-8f16-62192c39ade9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:13:08.208543 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4e4c79-bbea-4779-8f16-62192c39ade9 admin admin] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:13:08.208997 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4e4c79-bbea-4779-8f16-62192c39ade9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:13:08.210158 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4e4c79-bbea-4779-8f16-62192c39ade9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:13:08.210833 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef4e4c79-bbea-4779-8f16-62192c39ade9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:13:08.212768 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef4e4c79-bbea-4779-8f16-62192c39ade9 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:13:08.250468 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1341/2664] 10.222.0.40 () {60 vars in 1153 bytes} [Fri Aug 26 11:13:08 2022] POST /identity/v3/projects => generated 361 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 11:13:08.256264 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6b6405-7625-4b91-8203-63bc12f53a3e admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:13:08.267189 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6b6405-7625-4b91-8203-63bc12f53a3e admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:13:08.339578 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9ec658-ce4a-4e1b-bb4c-157f9f6a4b46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:13:08.340602 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9ec658-ce4a-4e1b-bb4c-157f9f6a4b46 admin admin] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:13:08.341043 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9ec658-ce4a-4e1b-bb4c-157f9f6a4b46 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:13:08.343614 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9ec658-ce4a-4e1b-bb4c-157f9f6a4b46 admin admin] PATH_INFO: `/v3/users` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:13:08.343614 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c9ec658-ce4a-4e1b-bb4c-157f9f6a4b46 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:13:08.347724 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c9ec658-ce4a-4e1b-bb4c-157f9f6a4b46 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:13:08.380278 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1324/2665] 10.222.0.40 () {60 vars in 1144 bytes} [Fri Aug 26 11:13:08 2022] POST /identity/v3/users => generated 281 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 26 11:13:08.381701 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4e4c79-bbea-4779-8f16-62192c39ade9 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:13:08.396011 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4e4c79-bbea-4779-8f16-62192c39ade9 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:13:08.404467 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02fc2704-bf10-428d-a8f1-3554f2655e3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:13:08.406613 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fc2704-bf10-428d-a8f1-3554f2655e3e admin admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:13:08.407433 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fc2704-bf10-428d-a8f1-3554f2655e3e admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:13:08.407858 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fc2704-bf10-428d-a8f1-3554f2655e3e admin admin] PATH_INFO: `/v3/roles` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:13:08.409558 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02fc2704-bf10-428d-a8f1-3554f2655e3e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:13:08.411535 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02fc2704-bf10-428d-a8f1-3554f2655e3e admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:13:08.433238 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1342/2666] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 26 11:13:08 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 11:13:08.438575 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9ec658-ce4a-4e1b-bb4c-157f9f6a4b46 admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:13:08.464110 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9ec658-ce4a-4e1b-bb4c-157f9f6a4b46 admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:13:08.472361 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fef6436-06bb-49f8-8cc8-74a17f949843 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:13:08.473271 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fef6436-06bb-49f8-8cc8-74a17f949843 admin admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:13:08.473708 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fef6436-06bb-49f8-8cc8-74a17f949843 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:13:08.474817 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fef6436-06bb-49f8-8cc8-74a17f949843 admin admin] PATH_INFO: `/v3/roles` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:13:08.475623 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fef6436-06bb-49f8-8cc8-74a17f949843 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:13:08.477661 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fef6436-06bb-49f8-8cc8-74a17f949843 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:13:08.508163 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1325/2667] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 26 11:13:08 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 11:13:08.512915 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02fc2704-bf10-428d-a8f1-3554f2655e3e admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:13:08.527900 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02fc2704-bf10-428d-a8f1-3554f2655e3e admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:13:08.541232 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c905ddeb-38b2-40bd-b42f-1d11ef4c3a95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:13:08.542604 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c905ddeb-38b2-40bd-b42f-1d11ef4c3a95 admin admin] REQUEST_METHOD: `PUT` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:13:08.543030 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c905ddeb-38b2-40bd-b42f-1d11ef4c3a95 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:13:08.543455 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c905ddeb-38b2-40bd-b42f-1d11ef4c3a95 admin admin] PATH_INFO: `/v3/projects/3805464816ac48018e56b4974e75381a/users/83507ef3ad334a1a84c0f2c6b1e6f9b8/roles/23e174ce743b46eabe618c212691da55` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:13:08.595935 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c905ddeb-38b2-40bd-b42f-1d11ef4c3a95 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3805464816ac48018e56b4974e75381a, user_id=83507ef3ad334a1a84c0f2c6b1e6f9b8, role_id=23e174ce743b46eabe618c212691da55)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:13:08.599779 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c905ddeb-38b2-40bd-b42f-1d11ef4c3a95 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:13:08.629094 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1343/2668] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 26 11:13:08 2022] PUT /identity/v3/projects/3805464816ac48018e56b4974e75381a/users/83507ef3ad334a1a84c0f2c6b1e6f9b8/roles/23e174ce743b46eabe618c212691da55 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 11:13:08.644249 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f75f6b9-1bb4-4721-9bfe-e9cd23d95ea9 None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:13:08.645494 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f75f6b9-1bb4-4721-9bfe-e9cd23d95ea9 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:13:08.645794 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f75f6b9-1bb4-4721-9bfe-e9cd23d95ea9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:13:08.703079 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-0f75f6b9-1bb4-4721-9bfe-e9cd23d95ea9 None None] MFA Rules not processed for user `83507ef3ad334a1a84c0f2c6b1e6f9b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 11:13:08.706486 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-0f75f6b9-1bb4-4721-9bfe-e9cd23d95ea9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:13:08.797417 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 11:13:08.824486 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1326/2669] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 11:13:08 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 11:13:08.867756 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c905ddeb-38b2-40bd-b42f-1d11ef4c3a95 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:13:08.869676 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-c905ddeb-38b2-40bd-b42f-1d11ef4c3a95 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:13:08.990390 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c905ddeb-38b2-40bd-b42f-1d11ef4c3a95 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:13:08.994156 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf8650a-268e-458d-8c91-3b76896d5c3c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c277338a7f814cc6b01c5419775031bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e30ccf0c1204727b786876c141dbcd2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:13:08.994767 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf8650a-268e-458d-8c91-3b76896d5c3c service cinder] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:13:08.995040 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf8650a-268e-458d-8c91-3b76896d5c3c service cinder] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:13:08.995308 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf8650a-268e-458d-8c91-3b76896d5c3c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:13:09.007287 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf8650a-268e-458d-8c91-3b76896d5c3c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:13:09.008752 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf8650a-268e-458d-8c91-3b76896d5c3c service cinder] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:13:09.018742 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1344/2670] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 11:13:08 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 11:13:09.034968 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f75f6b9-1bb4-4721-9bfe-e9cd23d95ea9 None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:13:09.122038 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f75f6b9-1bb4-4721-9bfe-e9cd23d95ea9 None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:13:09.130814 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ba540b-2789-4f3c-ad53-6494cb447c64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:13:09.131416 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ba540b-2789-4f3c-ad53-6494cb447c64 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:13:09.131694 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ba540b-2789-4f3c-ad53-6494cb447c64 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:13:09.131991 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ba540b-2789-4f3c-ad53-6494cb447c64 admin admin] PATH_INFO: `/v3/users/83507ef3ad334a1a84c0f2c6b1e6f9b8` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:13:09.133286 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ba540b-2789-4f3c-ad53-6494cb447c64 admin admin] RBAC: Authorizing `identity:delete_user(user_id=83507ef3ad334a1a84c0f2c6b1e6f9b8)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:13:09.135270 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ba540b-2789-4f3c-ad53-6494cb447c64 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:13:09.195349 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-00ba540b-2789-4f3c-ad53-6494cb447c64 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '83507ef3ad334a1a84c0f2c6b1e6f9b8'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:13:09.210815 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-00ba540b-2789-4f3c-ad53-6494cb447c64 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '83507ef3ad334a1a84c0f2c6b1e6f9b8'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:13:09.211905 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-00ba540b-2789-4f3c-ad53-6494cb447c64 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '83507ef3ad334a1a84c0f2c6b1e6f9b8'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:13:09.228842 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-00ba540b-2789-4f3c-ad53-6494cb447c64 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '83507ef3ad334a1a84c0f2c6b1e6f9b8'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:13:09.229854 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-00ba540b-2789-4f3c-ad53-6494cb447c64 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '83507ef3ad334a1a84c0f2c6b1e6f9b8'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:13:09.250471 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1327/2671] 10.222.0.40 () {58 vars in 1224 bytes} [Fri Aug 26 11:13:09 2022] DELETE /identity/v3/users/83507ef3ad334a1a84c0f2c6b1e6f9b8 => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 11:13:09.256422 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904f22b3-116c-4740-a2a0-8e288730556e None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:13:09.256957 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904f22b3-116c-4740-a2a0-8e288730556e None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:13:09.257081 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904f22b3-116c-4740-a2a0-8e288730556e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:13:09.272062 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 11:13:09.278296 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-904f22b3-116c-4740-a2a0-8e288730556e None None] MFA Rules not processed for user `ca155a48e2024c32af3fd2e19e5fac11`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 11:13:09.280376 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-904f22b3-116c-4740-a2a0-8e288730556e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:13:09.362163 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1345/2672] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 11:13:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 11:13:09.372893 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ba540b-2789-4f3c-ad53-6494cb447c64 admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:13:09.384310 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-00ba540b-2789-4f3c-ad53-6494cb447c64 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:13:09.449214 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ba540b-2789-4f3c-ad53-6494cb447c64 admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:13:09.454190 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9f7fa1-8aad-4a09-ba1a-aeeb7c2b0bb5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91da2acd2534423091537e1ba068d2b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e30ccf0c1204727b786876c141dbcd2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:13:09.454908 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9f7fa1-8aad-4a09-ba1a-aeeb7c2b0bb5 service neutron] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:13:09.455182 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9f7fa1-8aad-4a09-ba1a-aeeb7c2b0bb5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:13:09.455456 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9f7fa1-8aad-4a09-ba1a-aeeb7c2b0bb5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:13:09.465640 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b9f7fa1-8aad-4a09-ba1a-aeeb7c2b0bb5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:13:09.467299 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b9f7fa1-8aad-4a09-ba1a-aeeb7c2b0bb5 service neutron] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:13:09.477861 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1328/2673] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 11:13:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 11:13:09.797505 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904f22b3-116c-4740-a2a0-8e288730556e None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:13:09.799416 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-904f22b3-116c-4740-a2a0-8e288730556e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:13:09.817098 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904f22b3-116c-4740-a2a0-8e288730556e None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:13:09.824750 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a360c48-cd64-4243-a536-fb9ff93b6354 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:13:09.825790 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a360c48-cd64-4243-a536-fb9ff93b6354 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:13:09.826301 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a360c48-cd64-4243-a536-fb9ff93b6354 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:13:09.826763 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a360c48-cd64-4243-a536-fb9ff93b6354 admin admin] PATH_INFO: `/v3/projects/3805464816ac48018e56b4974e75381a` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:13:09.828423 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a360c48-cd64-4243-a536-fb9ff93b6354 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3805464816ac48018e56b4974e75381a)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:13:09.831825 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a360c48-cd64-4243-a536-fb9ff93b6354 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:13:09.938546 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-8a360c48-cd64-4243-a536-fb9ff93b6354 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3805464816ac48018e56b4974e75381a', 'request_id': 'req-8a360c48-cd64-4243-a536-fb9ff93b6354'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:13:09.946792 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-8a360c48-cd64-4243-a536-fb9ff93b6354 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3805464816ac48018e56b4974e75381a', 'request_id': 'req-8a360c48-cd64-4243-a536-fb9ff93b6354'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:13:09.956934 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-8a360c48-cd64-4243-a536-fb9ff93b6354 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3805464816ac48018e56b4974e75381a', 'request_id': 'req-8a360c48-cd64-4243-a536-fb9ff93b6354'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:13:09.981834 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-8a360c48-cd64-4243-a536-fb9ff93b6354 admin admin] The token cache is being invalidate because project 3805464816ac48018e56b4974e75381a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81424) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 11:13:09.982448 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-8a360c48-cd64-4243-a536-fb9ff93b6354 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:13:09.983731 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-8a360c48-cd64-4243-a536-fb9ff93b6354 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:13:09.990077 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1346/2674] 10.222.0.40 () {58 vars in 1233 bytes} [Fri Aug 26 11:13:09 2022] DELETE /identity/v3/projects/3805464816ac48018e56b4974e75381a => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 26 11:13:09.994815 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0c4b38-368a-40a0-9716-fc366c7a68dd None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:13:09.995316 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0c4b38-368a-40a0-9716-fc366c7a68dd None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:13:09.995697 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0c4b38-368a-40a0-9716-fc366c7a68dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:13:10.038476 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-cc0c4b38-368a-40a0-9716-fc366c7a68dd None None] MFA Rules not processed for user `ca155a48e2024c32af3fd2e19e5fac11`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 11:13:10.041252 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-cc0c4b38-368a-40a0-9716-fc366c7a68dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:13:10.059621 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 11:13:10.125594 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1329/2675] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 11:13:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 11:13:10.133605 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24262f48-1745-44e1-bf02-27f52cfb860c None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:13:10.133914 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24262f48-1745-44e1-bf02-27f52cfb860c None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:13:10.134332 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24262f48-1745-44e1-bf02-27f52cfb860c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:13:10.181830 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-24262f48-1745-44e1-bf02-27f52cfb860c None None] MFA Rules not processed for user `ca155a48e2024c32af3fd2e19e5fac11`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 11:13:10.183913 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-24262f48-1745-44e1-bf02-27f52cfb860c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:13:10.196532 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1347/2676] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 11:13:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 64 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 11:13:10.201164 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0c4b38-368a-40a0-9716-fc366c7a68dd None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:13:10.212540 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0c4b38-368a-40a0-9716-fc366c7a68dd None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:13:10.220316 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-262cd380-ebf3-4964-9914-501325fd9ae2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:13:10.221327 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262cd380-ebf3-4964-9914-501325fd9ae2 admin admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:13:10.221764 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262cd380-ebf3-4964-9914-501325fd9ae2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:13:10.222184 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262cd380-ebf3-4964-9914-501325fd9ae2 admin admin] PATH_INFO: `/v3/domains` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:13:10.226183 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-262cd380-ebf3-4964-9914-501325fd9ae2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 11:13:10.226183 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-262cd380-ebf3-4964-9914-501325fd9ae2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:13:10.226183 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-262cd380-ebf3-4964-9914-501325fd9ae2 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:13:10.241171 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1330/2677] 10.222.0.40 () {58 vars in 1153 bytes} [Fri Aug 26 11:13:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 11:13:10.245665 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24262f48-1745-44e1-bf02-27f52cfb860c None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:13:10.256797 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24262f48-1745-44e1-bf02-27f52cfb860c None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:13:10.277464 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6335d21a-0d6d-4869-bd26-65c550227cdf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:13:10.278183 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6335d21a-0d6d-4869-bd26-65c550227cdf admin admin] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:13:10.279352 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6335d21a-0d6d-4869-bd26-65c550227cdf admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:13:10.279812 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6335d21a-0d6d-4869-bd26-65c550227cdf admin admin] PATH_INFO: `/v3/projects` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:13:10.280900 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6335d21a-0d6d-4869-bd26-65c550227cdf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:13:10.283329 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6335d21a-0d6d-4869-bd26-65c550227cdf admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:13:10.322195 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1348/2678] 10.222.0.40 () {60 vars in 1153 bytes} [Fri Aug 26 11:13:10 2022] POST /identity/v3/projects => generated 375 bytes in 77 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 26 11:13:10.327371 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-262cd380-ebf3-4964-9914-501325fd9ae2 admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:13:10.338919 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-262cd380-ebf3-4964-9914-501325fd9ae2 admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:13:10.427728 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2233e4-986e-4ec9-a2f9-7aad5cacc79b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:13:10.428538 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2233e4-986e-4ec9-a2f9-7aad5cacc79b admin admin] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:13:10.428908 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2233e4-986e-4ec9-a2f9-7aad5cacc79b admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:13:10.429192 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2233e4-986e-4ec9-a2f9-7aad5cacc79b admin admin] PATH_INFO: `/v3/users` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:13:10.429830 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d2233e4-986e-4ec9-a2f9-7aad5cacc79b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:13:10.431826 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d2233e4-986e-4ec9-a2f9-7aad5cacc79b admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:13:10.455629 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1331/2679] 10.222.0.40 () {60 vars in 1144 bytes} [Fri Aug 26 11:13:10 2022] POST /identity/v3/users => generated 288 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 26 11:13:10.460113 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6335d21a-0d6d-4869-bd26-65c550227cdf admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:13:10.473438 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6335d21a-0d6d-4869-bd26-65c550227cdf admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:13:10.479166 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8668e4d-3366-4a40-85b9-d8048ef6f96d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:13:10.479788 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8668e4d-3366-4a40-85b9-d8048ef6f96d admin admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:13:10.480072 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8668e4d-3366-4a40-85b9-d8048ef6f96d admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:13:10.480345 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8668e4d-3366-4a40-85b9-d8048ef6f96d admin admin] PATH_INFO: `/v3/roles` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:13:10.480991 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8668e4d-3366-4a40-85b9-d8048ef6f96d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:13:10.482719 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8668e4d-3366-4a40-85b9-d8048ef6f96d admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:13:10.493457 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1349/2680] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 26 11:13:10 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 11:13:10.497560 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2233e4-986e-4ec9-a2f9-7aad5cacc79b admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:13:10.508770 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2233e4-986e-4ec9-a2f9-7aad5cacc79b admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:13:10.514655 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b710d5ea-887c-4711-89ae-a9819514abef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:13:10.515270 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b710d5ea-887c-4711-89ae-a9819514abef admin admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:13:10.515545 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b710d5ea-887c-4711-89ae-a9819514abef admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:13:10.515819 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b710d5ea-887c-4711-89ae-a9819514abef admin admin] PATH_INFO: `/v3/roles` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:13:10.516471 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b710d5ea-887c-4711-89ae-a9819514abef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:13:10.518109 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b710d5ea-887c-4711-89ae-a9819514abef admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:13:10.530494 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1332/2681] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 26 11:13:10 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 11:13:10.534848 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8668e4d-3366-4a40-85b9-d8048ef6f96d admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:13:10.545537 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8668e4d-3366-4a40-85b9-d8048ef6f96d admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:13:10.550269 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1ac10e-4660-43ab-aa87-39fbec2d81e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:13:10.550923 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1ac10e-4660-43ab-aa87-39fbec2d81e2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:13:10.551196 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1ac10e-4660-43ab-aa87-39fbec2d81e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:13:10.551465 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1ac10e-4660-43ab-aa87-39fbec2d81e2 admin admin] PATH_INFO: `/v3/projects/4d095f6e209444d7a59af31e9c76e878/users/09759b40664a4c83955550092d11ef63/roles/23e174ce743b46eabe618c212691da55` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:13:10.568005 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d1ac10e-4660-43ab-aa87-39fbec2d81e2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4d095f6e209444d7a59af31e9c76e878, user_id=09759b40664a4c83955550092d11ef63, role_id=23e174ce743b46eabe618c212691da55)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:13:10.570859 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d1ac10e-4660-43ab-aa87-39fbec2d81e2 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:13:10.585425 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1350/2682] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 26 11:13:10 2022] PUT /identity/v3/projects/4d095f6e209444d7a59af31e9c76e878/users/09759b40664a4c83955550092d11ef63/roles/23e174ce743b46eabe618c212691da55 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 11:13:10.597598 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4e1fa1-37e5-4ae2-958c-d1e814210b5a None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:13:10.597878 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4e1fa1-37e5-4ae2-958c-d1e814210b5a None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:13:10.598222 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4e1fa1-37e5-4ae2-958c-d1e814210b5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:13:10.628170 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-cc4e1fa1-37e5-4ae2-958c-d1e814210b5a None None] MFA Rules not processed for user `09759b40664a4c83955550092d11ef63`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 11:13:10.629413 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-cc4e1fa1-37e5-4ae2-958c-d1e814210b5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:13:10.697630 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1333/2683] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 11:13:10 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 11:13:10.702903 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1ac10e-4660-43ab-aa87-39fbec2d81e2 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:13:10.735192 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 11:13:10.773483 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1ac10e-4660-43ab-aa87-39fbec2d81e2 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:13:10.779557 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd6c957-dfc8-4aa0-83c2-9467ae068328 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:13:10.780210 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd6c957-dfc8-4aa0-83c2-9467ae068328 admin admin] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:13:10.780483 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd6c957-dfc8-4aa0-83c2-9467ae068328 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:13:10.780749 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd6c957-dfc8-4aa0-83c2-9467ae068328 admin admin] PATH_INFO: `/v3/projects` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:13:10.781405 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd6c957-dfc8-4aa0-83c2-9467ae068328 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:13:10.783371 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd6c957-dfc8-4aa0-83c2-9467ae068328 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:13:10.814606 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1351/2684] 10.222.0.40 () {60 vars in 1153 bytes} [Fri Aug 26 11:13:10 2022] POST /identity/v3/projects => generated 375 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 11:13:10.818920 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4e1fa1-37e5-4ae2-958c-d1e814210b5a None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:13:10.831501 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4e1fa1-37e5-4ae2-958c-d1e814210b5a None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:13:10.841479 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 11:13:10.899007 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c616b8a2-42a1-4413-b5ff-c7b527b8d10a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:13:10.899632 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c616b8a2-42a1-4413-b5ff-c7b527b8d10a admin admin] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:13:10.899909 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c616b8a2-42a1-4413-b5ff-c7b527b8d10a admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:13:10.900189 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c616b8a2-42a1-4413-b5ff-c7b527b8d10a admin admin] PATH_INFO: `/v3/users` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:13:10.900956 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c616b8a2-42a1-4413-b5ff-c7b527b8d10a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:13:10.903293 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c616b8a2-42a1-4413-b5ff-c7b527b8d10a admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:13:10.927604 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1334/2685] 10.222.0.40 () {60 vars in 1144 bytes} [Fri Aug 26 11:13:10 2022] POST /identity/v3/users => generated 288 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 11:13:10.931773 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd6c957-dfc8-4aa0-83c2-9467ae068328 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:13:10.943054 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd6c957-dfc8-4aa0-83c2-9467ae068328 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:13:10.947844 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a01505-923a-4112-87b9-b7bbfc9fb576 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:13:10.948454 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a01505-923a-4112-87b9-b7bbfc9fb576 admin admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:13:10.948735 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a01505-923a-4112-87b9-b7bbfc9fb576 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:13:10.949027 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a01505-923a-4112-87b9-b7bbfc9fb576 admin admin] PATH_INFO: `/v3/roles` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:13:10.949672 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a01505-923a-4112-87b9-b7bbfc9fb576 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:13:10.952030 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a01505-923a-4112-87b9-b7bbfc9fb576 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:13:10.963361 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1352/2686] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 26 11:13:10 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 26 11:13:10.967641 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c616b8a2-42a1-4413-b5ff-c7b527b8d10a admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:13:10.978089 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c616b8a2-42a1-4413-b5ff-c7b527b8d10a admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:13:10.983610 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf25313-b485-485c-a9b7-50a7aece59d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:13:10.984222 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf25313-b485-485c-a9b7-50a7aece59d5 admin admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:13:10.984499 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf25313-b485-485c-a9b7-50a7aece59d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:13:10.984771 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf25313-b485-485c-a9b7-50a7aece59d5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:13:10.985480 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf25313-b485-485c-a9b7-50a7aece59d5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:13:10.987060 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf25313-b485-485c-a9b7-50a7aece59d5 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:13:10.998753 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1335/2687] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 26 11:13:10 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 11:13:11.003229 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a01505-923a-4112-87b9-b7bbfc9fb576 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:13:11.013726 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a01505-923a-4112-87b9-b7bbfc9fb576 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:13:11.019426 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef3ce38-ac17-4e93-97dc-b9f0eabc15f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:13:11.020090 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef3ce38-ac17-4e93-97dc-b9f0eabc15f5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:13:11.020370 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef3ce38-ac17-4e93-97dc-b9f0eabc15f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:13:11.020651 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef3ce38-ac17-4e93-97dc-b9f0eabc15f5 admin admin] PATH_INFO: `/v3/projects/f902ed912f434d7cb022d6198e65e828/users/4534a9afd6a94c6fb992c5e3f7e274d6/roles/23e174ce743b46eabe618c212691da55` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:13:11.036788 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef3ce38-ac17-4e93-97dc-b9f0eabc15f5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f902ed912f434d7cb022d6198e65e828, user_id=4534a9afd6a94c6fb992c5e3f7e274d6, role_id=23e174ce743b46eabe618c212691da55)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:13:11.039507 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef3ce38-ac17-4e93-97dc-b9f0eabc15f5 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:13:11.053824 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1353/2688] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 26 11:13:11 2022] PUT /identity/v3/projects/f902ed912f434d7cb022d6198e65e828/users/4534a9afd6a94c6fb992c5e3f7e274d6/roles/23e174ce743b46eabe618c212691da55 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 11:13:11.065091 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c443c68-e83f-487a-b7e6-ba01bfa4ab96 None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:13:11.065631 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c443c68-e83f-487a-b7e6-ba01bfa4ab96 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:13:11.065730 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c443c68-e83f-487a-b7e6-ba01bfa4ab96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:13:11.100193 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-1c443c68-e83f-487a-b7e6-ba01bfa4ab96 None None] MFA Rules not processed for user `4534a9afd6a94c6fb992c5e3f7e274d6`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 11:13:11.101305 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-1c443c68-e83f-487a-b7e6-ba01bfa4ab96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:13:11.188663 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1336/2689] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 11:13:11 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 11:13:11.193214 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef3ce38-ac17-4e93-97dc-b9f0eabc15f5 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:13:11.271451 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef3ce38-ac17-4e93-97dc-b9f0eabc15f5 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:13:11.276893 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb32e0bd-064d-441b-ad29-48a17268eb5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:13:11.277516 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb32e0bd-064d-441b-ad29-48a17268eb5c admin admin] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:13:11.277780 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb32e0bd-064d-441b-ad29-48a17268eb5c admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:13:11.278082 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb32e0bd-064d-441b-ad29-48a17268eb5c admin admin] PATH_INFO: `/v3/projects` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:13:11.279334 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb32e0bd-064d-441b-ad29-48a17268eb5c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:13:11.283631 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb32e0bd-064d-441b-ad29-48a17268eb5c admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:13:11.316189 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1354/2690] 10.222.0.40 () {60 vars in 1153 bytes} [Fri Aug 26 11:13:11 2022] POST /identity/v3/projects => generated 375 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 11:13:11.321031 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c443c68-e83f-487a-b7e6-ba01bfa4ab96 None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:13:11.334913 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c443c68-e83f-487a-b7e6-ba01bfa4ab96 None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:13:11.400850 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7e2e6c-1f97-4904-80a4-5580877a36b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:13:11.401474 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7e2e6c-1f97-4904-80a4-5580877a36b6 admin admin] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:13:11.401929 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7e2e6c-1f97-4904-80a4-5580877a36b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:13:11.402277 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7e2e6c-1f97-4904-80a4-5580877a36b6 admin admin] PATH_INFO: `/v3/users` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:13:11.402899 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff7e2e6c-1f97-4904-80a4-5580877a36b6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:13:11.405124 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff7e2e6c-1f97-4904-80a4-5580877a36b6 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:13:11.427916 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1337/2691] 10.222.0.40 () {60 vars in 1144 bytes} [Fri Aug 26 11:13:11 2022] POST /identity/v3/users => generated 294 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 11:13:11.431767 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb32e0bd-064d-441b-ad29-48a17268eb5c admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:13:11.443536 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb32e0bd-064d-441b-ad29-48a17268eb5c admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:13:11.448250 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6904d5ea-4798-4301-92c0-7563de84e1b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:13:11.448859 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6904d5ea-4798-4301-92c0-7563de84e1b7 admin admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:13:11.449134 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6904d5ea-4798-4301-92c0-7563de84e1b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:13:11.449411 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6904d5ea-4798-4301-92c0-7563de84e1b7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:13:11.450086 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6904d5ea-4798-4301-92c0-7563de84e1b7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:13:11.452350 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6904d5ea-4798-4301-92c0-7563de84e1b7 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:13:11.463772 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1355/2692] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 26 11:13:11 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 11:13:11.468129 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7e2e6c-1f97-4904-80a4-5580877a36b6 admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:13:11.478504 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7e2e6c-1f97-4904-80a4-5580877a36b6 admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:13:11.483626 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b5049f-109e-4ed6-b067-36ba7fcceaf4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:13:11.484283 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b5049f-109e-4ed6-b067-36ba7fcceaf4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:13:11.484561 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b5049f-109e-4ed6-b067-36ba7fcceaf4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:13:11.484832 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b5049f-109e-4ed6-b067-36ba7fcceaf4 admin admin] PATH_INFO: `/v3/projects/8c5db902e9b54e64ab25ba324cd16ae5/users/a9120d7ae3664896b1aafd9dc6afb76d/roles/1f97e4602e2e4d6498913a81f9ae8d6a` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:13:11.493802 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 11:13:11.503001 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0b5049f-109e-4ed6-b067-36ba7fcceaf4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8c5db902e9b54e64ab25ba324cd16ae5, user_id=a9120d7ae3664896b1aafd9dc6afb76d, role_id=1f97e4602e2e4d6498913a81f9ae8d6a)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:13:11.505955 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0b5049f-109e-4ed6-b067-36ba7fcceaf4 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:13:11.520433 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1338/2693] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 26 11:13:11 2022] PUT /identity/v3/projects/8c5db902e9b54e64ab25ba324cd16ae5/users/a9120d7ae3664896b1aafd9dc6afb76d/roles/1f97e4602e2e4d6498913a81f9ae8d6a => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 26 11:13:11.525133 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6904d5ea-4798-4301-92c0-7563de84e1b7 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:13:11.536334 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6904d5ea-4798-4301-92c0-7563de84e1b7 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:13:11.587375 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 11:13:11.599488 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e4be71-05c7-4977-961f-f4eafb7778b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:13:11.600101 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e4be71-05c7-4977-961f-f4eafb7778b2 admin admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:13:11.600374 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e4be71-05c7-4977-961f-f4eafb7778b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:13:11.600640 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e4be71-05c7-4977-961f-f4eafb7778b2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:13:11.601278 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e4be71-05c7-4977-961f-f4eafb7778b2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:13:11.602909 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e4be71-05c7-4977-961f-f4eafb7778b2 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:13:11.614717 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1356/2694] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 26 11:13:11 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 11:13:11.619433 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b5049f-109e-4ed6-b067-36ba7fcceaf4 admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:13:11.630955 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b5049f-109e-4ed6-b067-36ba7fcceaf4 admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:13:11.636969 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096668db-0599-4eaf-b3c4-dee47ef2782a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:13:11.637629 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096668db-0599-4eaf-b3c4-dee47ef2782a admin admin] REQUEST_METHOD: `PUT` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:13:11.638522 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096668db-0599-4eaf-b3c4-dee47ef2782a admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:13:11.638804 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096668db-0599-4eaf-b3c4-dee47ef2782a admin admin] PATH_INFO: `/v3/projects/8c5db902e9b54e64ab25ba324cd16ae5/users/a9120d7ae3664896b1aafd9dc6afb76d/roles/1f97e4602e2e4d6498913a81f9ae8d6a` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:13:11.641394 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-096668db-0599-4eaf-b3c4-dee47ef2782a admin admin] RBAC: Authorizing `identity:create_grant(project_id=8c5db902e9b54e64ab25ba324cd16ae5, user_id=a9120d7ae3664896b1aafd9dc6afb76d, role_id=1f97e4602e2e4d6498913a81f9ae8d6a)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:13:11.644086 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-096668db-0599-4eaf-b3c4-dee47ef2782a admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:13:11.655327 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1339/2695] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 26 11:13:11 2022] PUT /identity/v3/projects/8c5db902e9b54e64ab25ba324cd16ae5/users/a9120d7ae3664896b1aafd9dc6afb76d/roles/1f97e4602e2e4d6498913a81f9ae8d6a => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 11:13:11.665831 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8563bbd7-6def-46cf-9411-27d45ae42577 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:13:11.666158 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8563bbd7-6def-46cf-9411-27d45ae42577 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:13:11.666439 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8563bbd7-6def-46cf-9411-27d45ae42577 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:13:11.696517 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-8563bbd7-6def-46cf-9411-27d45ae42577 None None] MFA Rules not processed for user `a9120d7ae3664896b1aafd9dc6afb76d`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 11:13:11.697745 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-8563bbd7-6def-46cf-9411-27d45ae42577 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:13:11.772255 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1357/2696] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 11:13:11 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 11:13:11.785012 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096668db-0599-4eaf-b3c4-dee47ef2782a admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:13:11.786832 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-096668db-0599-4eaf-b3c4-dee47ef2782a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:13:11.850362 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096668db-0599-4eaf-b3c4-dee47ef2782a admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:13:11.854138 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2f5b2b-7ec5-4334-8684-c14aaa69149f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c277338a7f814cc6b01c5419775031bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e30ccf0c1204727b786876c141dbcd2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:13:11.854747 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2f5b2b-7ec5-4334-8684-c14aaa69149f service cinder] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:13:11.855021 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2f5b2b-7ec5-4334-8684-c14aaa69149f service cinder] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:13:11.855284 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2f5b2b-7ec5-4334-8684-c14aaa69149f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:13:11.916668 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e2f5b2b-7ec5-4334-8684-c14aaa69149f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:13:11.918148 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e2f5b2b-7ec5-4334-8684-c14aaa69149f service cinder] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:13:11.928140 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1340/2697] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 11:13:11 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 11:13:13.503032 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8563bbd7-6def-46cf-9411-27d45ae42577 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:13:13.504427 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-8563bbd7-6def-46cf-9411-27d45ae42577 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:13:13.518739 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8563bbd7-6def-46cf-9411-27d45ae42577 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:13:13.522726 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbee9410-8b84-44cc-8108-1c1504850203 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c277338a7f814cc6b01c5419775031bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e30ccf0c1204727b786876c141dbcd2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:13:13.523430 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbee9410-8b84-44cc-8108-1c1504850203 service cinder] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:13:13.523794 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbee9410-8b84-44cc-8108-1c1504850203 service cinder] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:13:13.524188 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbee9410-8b84-44cc-8108-1c1504850203 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:13:13.609532 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbee9410-8b84-44cc-8108-1c1504850203 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:13:13.612319 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbee9410-8b84-44cc-8108-1c1504850203 service cinder] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:13:13.627951 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1358/2698] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 11:13:13 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 11:13:14.200121 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2f5b2b-7ec5-4334-8684-c14aaa69149f service cinder] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:13:14.209924 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2f5b2b-7ec5-4334-8684-c14aaa69149f service cinder] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:13:14.213682 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc70fd3d-c9d1-41dc-be16-2b952ba65140 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c277338a7f814cc6b01c5419775031bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e30ccf0c1204727b786876c141dbcd2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:13:14.214322 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc70fd3d-c9d1-41dc-be16-2b952ba65140 service cinder] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:13:14.214600 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc70fd3d-c9d1-41dc-be16-2b952ba65140 service cinder] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:13:14.214904 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc70fd3d-c9d1-41dc-be16-2b952ba65140 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:13:14.224317 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc70fd3d-c9d1-41dc-be16-2b952ba65140 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:13:14.225772 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc70fd3d-c9d1-41dc-be16-2b952ba65140 service cinder] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:13:14.236157 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1341/2699] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 11:13:14 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 11:13:16.760479 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a7e205-aebe-48c3-b766-9e5723893700 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:13:16.761018 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a7e205-aebe-48c3-b766-9e5723893700 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:13:16.761121 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a7e205-aebe-48c3-b766-9e5723893700 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:13:16.781013 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-75a7e205-aebe-48c3-b766-9e5723893700 None None] MFA Rules not processed for user `b4ea31e78ceb46f8bc92d86211da5072`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 11:13:16.782260 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-75a7e205-aebe-48c3-b766-9e5723893700 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:13:16.842370 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1359/2700] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 11:13:16 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 11:13:18.234056 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc70fd3d-c9d1-41dc-be16-2b952ba65140 service cinder] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:13:18.314982 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc70fd3d-c9d1-41dc-be16-2b952ba65140 service cinder] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:13:18.330740 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a69112f-94c3-4bca-8460-ad52236a1376 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:13:18.331857 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a69112f-94c3-4bca-8460-ad52236a1376 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:13:18.332285 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a69112f-94c3-4bca-8460-ad52236a1376 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:13:18.332626 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a69112f-94c3-4bca-8460-ad52236a1376 admin admin] PATH_INFO: `/v3/users/09759b40664a4c83955550092d11ef63` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:13:18.334835 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a69112f-94c3-4bca-8460-ad52236a1376 admin admin] RBAC: Authorizing `identity:delete_user(user_id=09759b40664a4c83955550092d11ef63)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:13:18.337068 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a69112f-94c3-4bca-8460-ad52236a1376 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:13:18.361163 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=97,keystone:INSERT=2,keystone:DELETE=1 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 11:13:18.416594 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-0a69112f-94c3-4bca-8460-ad52236a1376 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '09759b40664a4c83955550092d11ef63'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:13:18.426450 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-0a69112f-94c3-4bca-8460-ad52236a1376 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '09759b40664a4c83955550092d11ef63'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:13:18.427735 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-0a69112f-94c3-4bca-8460-ad52236a1376 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '09759b40664a4c83955550092d11ef63'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:13:18.445420 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-0a69112f-94c3-4bca-8460-ad52236a1376 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '09759b40664a4c83955550092d11ef63'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:13:18.446390 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-0a69112f-94c3-4bca-8460-ad52236a1376 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '09759b40664a4c83955550092d11ef63'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:13:18.463789 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1342/2701] 10.222.0.40 () {58 vars in 1224 bytes} [Fri Aug 26 11:13:18 2022] DELETE /identity/v3/users/09759b40664a4c83955550092d11ef63 => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 11:13:18.474269 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cae665f-60f8-41ab-8cca-9cbb63b6023b None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:13:18.475164 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cae665f-60f8-41ab-8cca-9cbb63b6023b None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:13:18.476045 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cae665f-60f8-41ab-8cca-9cbb63b6023b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:13:18.509473 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-3cae665f-60f8-41ab-8cca-9cbb63b6023b None None] MFA Rules not processed for user `ca155a48e2024c32af3fd2e19e5fac11`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 11:13:18.511251 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-3cae665f-60f8-41ab-8cca-9cbb63b6023b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:13:18.535168 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 11:13:18.595379 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1360/2702] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 11:13:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 11:13:18.605110 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a69112f-94c3-4bca-8460-ad52236a1376 admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:13:18.607136 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-0a69112f-94c3-4bca-8460-ad52236a1376 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:13:18.672127 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a69112f-94c3-4bca-8460-ad52236a1376 admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:13:18.675742 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8217d46-d6d3-4fc5-b057-6e8f22bc5889 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91da2acd2534423091537e1ba068d2b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e30ccf0c1204727b786876c141dbcd2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:13:18.676361 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8217d46-d6d3-4fc5-b057-6e8f22bc5889 service neutron] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:13:18.676632 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8217d46-d6d3-4fc5-b057-6e8f22bc5889 service neutron] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:13:18.676899 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8217d46-d6d3-4fc5-b057-6e8f22bc5889 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:13:18.688720 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8217d46-d6d3-4fc5-b057-6e8f22bc5889 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:13:18.690190 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8217d46-d6d3-4fc5-b057-6e8f22bc5889 service neutron] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:13:18.701534 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1343/2703] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 11:13:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 11:13:19.050623 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cae665f-60f8-41ab-8cca-9cbb63b6023b None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:13:19.052358 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-3cae665f-60f8-41ab-8cca-9cbb63b6023b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:13:19.072401 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cae665f-60f8-41ab-8cca-9cbb63b6023b None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:13:19.084621 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70a4913-f022-4064-aa99-30460caae574 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:13:19.085371 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70a4913-f022-4064-aa99-30460caae574 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:13:19.085550 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70a4913-f022-4064-aa99-30460caae574 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:13:19.085822 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70a4913-f022-4064-aa99-30460caae574 admin admin] PATH_INFO: `/v3/projects/4d095f6e209444d7a59af31e9c76e878` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:13:19.093904 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c70a4913-f022-4064-aa99-30460caae574 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4d095f6e209444d7a59af31e9c76e878)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:13:19.095903 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c70a4913-f022-4064-aa99-30460caae574 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:13:19.156075 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-c70a4913-f022-4064-aa99-30460caae574 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4d095f6e209444d7a59af31e9c76e878', 'request_id': 'req-c70a4913-f022-4064-aa99-30460caae574'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:13:19.163806 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-c70a4913-f022-4064-aa99-30460caae574 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4d095f6e209444d7a59af31e9c76e878', 'request_id': 'req-c70a4913-f022-4064-aa99-30460caae574'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:13:19.170867 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-c70a4913-f022-4064-aa99-30460caae574 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4d095f6e209444d7a59af31e9c76e878', 'request_id': 'req-c70a4913-f022-4064-aa99-30460caae574'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:13:19.185462 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-c70a4913-f022-4064-aa99-30460caae574 admin admin] The token cache is being invalidate because project 4d095f6e209444d7a59af31e9c76e878 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81424) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 11:13:19.185707 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-c70a4913-f022-4064-aa99-30460caae574 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:13:19.186329 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-c70a4913-f022-4064-aa99-30460caae574 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:13:19.187988 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1361/2704] 10.222.0.40 () {58 vars in 1233 bytes} [Fri Aug 26 11:13:19 2022] DELETE /identity/v3/projects/4d095f6e209444d7a59af31e9c76e878 => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 11:13:19.191961 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8217d46-d6d3-4fc5-b057-6e8f22bc5889 service neutron] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:13:19.202325 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8217d46-d6d3-4fc5-b057-6e8f22bc5889 service neutron] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:13:19.265194 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558da24f-86d6-4e85-a699-8335f177570c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:13:19.265840 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558da24f-86d6-4e85-a699-8335f177570c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:13:19.266288 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558da24f-86d6-4e85-a699-8335f177570c admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:13:19.266561 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558da24f-86d6-4e85-a699-8335f177570c admin admin] PATH_INFO: `/v3/users/4534a9afd6a94c6fb992c5e3f7e274d6` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:13:19.268054 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-558da24f-86d6-4e85-a699-8335f177570c admin admin] RBAC: Authorizing `identity:delete_user(user_id=4534a9afd6a94c6fb992c5e3f7e274d6)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:13:19.269995 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-558da24f-86d6-4e85-a699-8335f177570c admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:13:19.314572 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:DELETE=14,keystone:SELECT=85,keystone:INSERT=1 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 11:13:19.322712 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-558da24f-86d6-4e85-a699-8335f177570c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4534a9afd6a94c6fb992c5e3f7e274d6'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:13:19.331847 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-558da24f-86d6-4e85-a699-8335f177570c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4534a9afd6a94c6fb992c5e3f7e274d6'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:13:19.332568 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-558da24f-86d6-4e85-a699-8335f177570c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4534a9afd6a94c6fb992c5e3f7e274d6'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:13:19.346210 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-558da24f-86d6-4e85-a699-8335f177570c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4534a9afd6a94c6fb992c5e3f7e274d6'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:13:19.346932 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-558da24f-86d6-4e85-a699-8335f177570c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4534a9afd6a94c6fb992c5e3f7e274d6'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:13:19.362150 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1344/2705] 10.222.0.40 () {58 vars in 1224 bytes} [Fri Aug 26 11:13:19 2022] DELETE /identity/v3/users/4534a9afd6a94c6fb992c5e3f7e274d6 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 11:13:19.664326 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70a4913-f022-4064-aa99-30460caae574 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:13:19.666902 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-c70a4913-f022-4064-aa99-30460caae574 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:13:19.690237 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70a4913-f022-4064-aa99-30460caae574 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:13:19.767775 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec515f22-f587-44a1-8b3d-edf933bbc6d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:13:19.768456 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec515f22-f587-44a1-8b3d-edf933bbc6d7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:13:19.768734 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec515f22-f587-44a1-8b3d-edf933bbc6d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:13:19.769268 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec515f22-f587-44a1-8b3d-edf933bbc6d7 admin admin] PATH_INFO: `/v3/projects/f902ed912f434d7cb022d6198e65e828` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:13:19.770496 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec515f22-f587-44a1-8b3d-edf933bbc6d7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f902ed912f434d7cb022d6198e65e828)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:13:19.772418 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec515f22-f587-44a1-8b3d-edf933bbc6d7 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:13:19.823174 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 11:13:19.834177 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-ec515f22-f587-44a1-8b3d-edf933bbc6d7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f902ed912f434d7cb022d6198e65e828', 'request_id': 'req-ec515f22-f587-44a1-8b3d-edf933bbc6d7'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:13:19.845601 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-ec515f22-f587-44a1-8b3d-edf933bbc6d7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f902ed912f434d7cb022d6198e65e828', 'request_id': 'req-ec515f22-f587-44a1-8b3d-edf933bbc6d7'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:13:19.852540 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-ec515f22-f587-44a1-8b3d-edf933bbc6d7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f902ed912f434d7cb022d6198e65e828', 'request_id': 'req-ec515f22-f587-44a1-8b3d-edf933bbc6d7'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:13:19.879362 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-ec515f22-f587-44a1-8b3d-edf933bbc6d7 admin admin] The token cache is being invalidate because project f902ed912f434d7cb022d6198e65e828 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81424) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 11:13:19.879683 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-ec515f22-f587-44a1-8b3d-edf933bbc6d7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:13:19.880823 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-ec515f22-f587-44a1-8b3d-edf933bbc6d7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:13:19.886407 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1362/2706] 10.222.0.40 () {58 vars in 1233 bytes} [Fri Aug 26 11:13:19 2022] DELETE /identity/v3/projects/f902ed912f434d7cb022d6198e65e828 => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 11:13:19.888418 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558da24f-86d6-4e85-a699-8335f177570c admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:13:19.889887 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-558da24f-86d6-4e85-a699-8335f177570c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:13:19.912975 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558da24f-86d6-4e85-a699-8335f177570c admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:13:19.985820 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842b3cf5-375b-4927-9052-fe58a51913a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:13:19.986903 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842b3cf5-375b-4927-9052-fe58a51913a9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:13:19.986903 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842b3cf5-375b-4927-9052-fe58a51913a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:13:19.987341 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842b3cf5-375b-4927-9052-fe58a51913a9 admin admin] PATH_INFO: `/v3/users/a9120d7ae3664896b1aafd9dc6afb76d` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:13:19.988938 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-842b3cf5-375b-4927-9052-fe58a51913a9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a9120d7ae3664896b1aafd9dc6afb76d)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:13:19.991234 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-842b3cf5-375b-4927-9052-fe58a51913a9 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:13:20.062626 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-842b3cf5-375b-4927-9052-fe58a51913a9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a9120d7ae3664896b1aafd9dc6afb76d'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:13:20.093000 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-842b3cf5-375b-4927-9052-fe58a51913a9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a9120d7ae3664896b1aafd9dc6afb76d'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:13:20.093789 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-842b3cf5-375b-4927-9052-fe58a51913a9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a9120d7ae3664896b1aafd9dc6afb76d'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:13:20.129059 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-842b3cf5-375b-4927-9052-fe58a51913a9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a9120d7ae3664896b1aafd9dc6afb76d'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:13:20.130315 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-842b3cf5-375b-4927-9052-fe58a51913a9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a9120d7ae3664896b1aafd9dc6afb76d'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:13:20.176768 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1345/2707] 10.222.0.40 () {58 vars in 1224 bytes} [Fri Aug 26 11:13:19 2022] DELETE /identity/v3/users/a9120d7ae3664896b1aafd9dc6afb76d => generated 0 bytes in 289 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 11:13:20.585896 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec515f22-f587-44a1-8b3d-edf933bbc6d7 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:13:20.587593 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-ec515f22-f587-44a1-8b3d-edf933bbc6d7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:13:20.618715 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec515f22-f587-44a1-8b3d-edf933bbc6d7 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:13:20.689066 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb74ba9-8d87-4cf6-a48c-867fb20093ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:13:20.689728 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb74ba9-8d87-4cf6-a48c-867fb20093ae admin admin] REQUEST_METHOD: `DELETE` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:13:20.689969 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb74ba9-8d87-4cf6-a48c-867fb20093ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:13:20.690264 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb74ba9-8d87-4cf6-a48c-867fb20093ae admin admin] PATH_INFO: `/v3/projects/8c5db902e9b54e64ab25ba324cd16ae5` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:13:20.691393 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb74ba9-8d87-4cf6-a48c-867fb20093ae admin admin] RBAC: Authorizing `identity:delete_project(project_id=8c5db902e9b54e64ab25ba324cd16ae5)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:13:20.693387 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb74ba9-8d87-4cf6-a48c-867fb20093ae admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:13:20.746515 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-feb74ba9-8d87-4cf6-a48c-867fb20093ae admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8c5db902e9b54e64ab25ba324cd16ae5', 'request_id': 'req-feb74ba9-8d87-4cf6-a48c-867fb20093ae'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:13:20.754219 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-feb74ba9-8d87-4cf6-a48c-867fb20093ae admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8c5db902e9b54e64ab25ba324cd16ae5', 'request_id': 'req-feb74ba9-8d87-4cf6-a48c-867fb20093ae'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:13:20.762205 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-feb74ba9-8d87-4cf6-a48c-867fb20093ae admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8c5db902e9b54e64ab25ba324cd16ae5', 'request_id': 'req-feb74ba9-8d87-4cf6-a48c-867fb20093ae'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:13:20.776797 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-feb74ba9-8d87-4cf6-a48c-867fb20093ae admin admin] The token cache is being invalidate because project 8c5db902e9b54e64ab25ba324cd16ae5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81424) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 11:13:20.777092 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-feb74ba9-8d87-4cf6-a48c-867fb20093ae admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:13:20.777761 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-feb74ba9-8d87-4cf6-a48c-867fb20093ae admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:13:20.794087 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1363/2708] 10.222.0.40 () {58 vars in 1233 bytes} [Fri Aug 26 11:13:20 2022] DELETE /identity/v3/projects/8c5db902e9b54e64ab25ba324cd16ae5 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 11:13:30.156185 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:DELETE=15,keystone:SELECT=67,keystone:INSERT=2 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 11:13:30.773464 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=60,keystone:DELETE=11,keystone:INSERT=2 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 11:13:38.706329 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7754a7b-8db7-4ebd-b74f-4f58e0603eaf None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:13:38.706329 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7754a7b-8db7-4ebd-b74f-4f58e0603eaf None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:13:38.706329 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7754a7b-8db7-4ebd-b74f-4f58e0603eaf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:13:38.738600 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-a7754a7b-8db7-4ebd-b74f-4f58e0603eaf None None] MFA Rules not processed for user `50be12ccbd274a8792615e9dd10b61ca`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 11:13:38.740113 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-a7754a7b-8db7-4ebd-b74f-4f58e0603eaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:13:38.812678 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1346/2709] 10.222.0.40 () {58 vars in 1009 bytes} [Fri Aug 26 11:13:38 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 11:13:38.863304 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb74ba9-8d87-4cf6-a48c-867fb20093ae admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:13:38.864754 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-feb74ba9-8d87-4cf6-a48c-867fb20093ae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:13:38.953660 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb74ba9-8d87-4cf6-a48c-867fb20093ae admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:13:38.958741 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa738ca-93d5-4332-9080-86ac6e85e43b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6529bea296c9482a973ced5fe9f2212f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e30ccf0c1204727b786876c141dbcd2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:13:38.959351 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa738ca-93d5-4332-9080-86ac6e85e43b service swift] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:13:38.959625 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa738ca-93d5-4332-9080-86ac6e85e43b service swift] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:13:38.959891 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa738ca-93d5-4332-9080-86ac6e85e43b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:13:38.969247 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa738ca-93d5-4332-9080-86ac6e85e43b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:13:38.970859 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa738ca-93d5-4332-9080-86ac6e85e43b service swift] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:13:38.979354 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1364/2710] 10.222.0.40 () {60 vars in 1384 bytes} [Fri Aug 26 11:13:38 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 26 11:13:48.805279 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=32 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 11:13:48.976356 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=32 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 11:13:55.315029 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5cdfb7-b658-491d-8923-1c9f30424f56 None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:13:55.315527 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5cdfb7-b658-491d-8923-1c9f30424f56 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:13:55.315641 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5cdfb7-b658-491d-8923-1c9f30424f56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:13:55.335383 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-ef5cdfb7-b658-491d-8923-1c9f30424f56 None None] MFA Rules not processed for user `50be12ccbd274a8792615e9dd10b61ca`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 11:13:55.336620 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-ef5cdfb7-b658-491d-8923-1c9f30424f56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:13:55.344813 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1347/2711] 10.222.0.40 () {58 vars in 967 bytes} [Fri Aug 26 11:13:55 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 11:13:55.359131 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa738ca-93d5-4332-9080-86ac6e85e43b service swift] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:13:55.370259 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa738ca-93d5-4332-9080-86ac6e85e43b service swift] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:13:55.376663 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cfb6004-e34f-4dba-9057-e87dce89ce4c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6529bea296c9482a973ced5fe9f2212f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e30ccf0c1204727b786876c141dbcd2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:13:55.377298 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cfb6004-e34f-4dba-9057-e87dce89ce4c service swift] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:13:55.377575 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cfb6004-e34f-4dba-9057-e87dce89ce4c service swift] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:13:55.377848 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cfb6004-e34f-4dba-9057-e87dce89ce4c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:13:55.386525 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cfb6004-e34f-4dba-9057-e87dce89ce4c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:13:55.388657 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cfb6004-e34f-4dba-9057-e87dce89ce4c service swift] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:13:55.397045 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1365/2712] 10.222.0.40 () {60 vars in 1384 bytes} [Fri Aug 26 11:13:55 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 26 11:13:57.498217 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5cdfb7-b658-491d-8923-1c9f30424f56 None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:13:57.501082 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-ef5cdfb7-b658-491d-8923-1c9f30424f56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:13:57.625884 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5cdfb7-b658-491d-8923-1c9f30424f56 None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:13:57.631813 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ffdf949-0509-420b-b41d-1b09eb4a37bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:13:57.632441 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffdf949-0509-420b-b41d-1b09eb4a37bd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:13:57.633043 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffdf949-0509-420b-b41d-1b09eb4a37bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:13:57.633043 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffdf949-0509-420b-b41d-1b09eb4a37bd admin admin] PATH_INFO: `/v3/users/b4ea31e78ceb46f8bc92d86211da5072` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:13:57.634343 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ffdf949-0509-420b-b41d-1b09eb4a37bd admin admin] RBAC: Authorizing `identity:delete_user(user_id=b4ea31e78ceb46f8bc92d86211da5072)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:13:57.636284 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ffdf949-0509-420b-b41d-1b09eb4a37bd admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:13:57.710011 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-6ffdf949-0509-420b-b41d-1b09eb4a37bd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b4ea31e78ceb46f8bc92d86211da5072'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:13:57.716793 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-6ffdf949-0509-420b-b41d-1b09eb4a37bd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b4ea31e78ceb46f8bc92d86211da5072'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:13:57.717613 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-6ffdf949-0509-420b-b41d-1b09eb4a37bd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b4ea31e78ceb46f8bc92d86211da5072'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:13:57.732775 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-6ffdf949-0509-420b-b41d-1b09eb4a37bd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b4ea31e78ceb46f8bc92d86211da5072'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:13:57.733618 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-6ffdf949-0509-420b-b41d-1b09eb4a37bd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b4ea31e78ceb46f8bc92d86211da5072'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:13:57.754808 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1348/2713] 10.222.0.40 () {58 vars in 1224 bytes} [Fri Aug 26 11:13:57 2022] DELETE /identity/v3/users/b4ea31e78ceb46f8bc92d86211da5072 => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 11:13:57.968063 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cfb6004-e34f-4dba-9057-e87dce89ce4c service swift] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:13:57.992655 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cfb6004-e34f-4dba-9057-e87dce89ce4c service swift] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:13:58.090852 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123503ec-d25f-4f90-b4ac-0cee9dd8ce40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:13:58.091703 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123503ec-d25f-4f90-b4ac-0cee9dd8ce40 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:13:58.092039 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123503ec-d25f-4f90-b4ac-0cee9dd8ce40 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:13:58.092331 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123503ec-d25f-4f90-b4ac-0cee9dd8ce40 admin admin] PATH_INFO: `/v3/projects/ef07a23929104d298d85485c33a0411e` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:13:58.093592 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-123503ec-d25f-4f90-b4ac-0cee9dd8ce40 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ef07a23929104d298d85485c33a0411e)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:13:58.095727 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-123503ec-d25f-4f90-b4ac-0cee9dd8ce40 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:13:58.164440 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-123503ec-d25f-4f90-b4ac-0cee9dd8ce40 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ef07a23929104d298d85485c33a0411e', 'request_id': 'req-123503ec-d25f-4f90-b4ac-0cee9dd8ce40'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:13:58.171786 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-123503ec-d25f-4f90-b4ac-0cee9dd8ce40 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ef07a23929104d298d85485c33a0411e', 'request_id': 'req-123503ec-d25f-4f90-b4ac-0cee9dd8ce40'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:13:58.182876 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-123503ec-d25f-4f90-b4ac-0cee9dd8ce40 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ef07a23929104d298d85485c33a0411e', 'request_id': 'req-123503ec-d25f-4f90-b4ac-0cee9dd8ce40'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:13:58.201728 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-123503ec-d25f-4f90-b4ac-0cee9dd8ce40 admin admin] The token cache is being invalidate because project ef07a23929104d298d85485c33a0411e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81424) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 11:13:58.202156 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-123503ec-d25f-4f90-b4ac-0cee9dd8ce40 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:13:58.202894 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-123503ec-d25f-4f90-b4ac-0cee9dd8ce40 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:13:58.204578 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1366/2714] 10.222.0.40 () {58 vars in 1233 bytes} [Fri Aug 26 11:13:57 2022] DELETE /identity/v3/projects/ef07a23929104d298d85485c33a0411e => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 11:13:58.209635 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c829d9d-e865-44f8-962d-4c8db99876fb None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:13:58.209949 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c829d9d-e865-44f8-962d-4c8db99876fb None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:13:58.210255 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c829d9d-e865-44f8-962d-4c8db99876fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:13:58.234332 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-3c829d9d-e865-44f8-962d-4c8db99876fb None None] MFA Rules not processed for user `ca155a48e2024c32af3fd2e19e5fac11`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 11:13:58.235492 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-3c829d9d-e865-44f8-962d-4c8db99876fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:13:58.303067 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1349/2715] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 11:13:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 11:13:58.369398 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85047317-e5dc-49b1-9ff3-d424d579ea27 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:13:58.369786 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85047317-e5dc-49b1-9ff3-d424d579ea27 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:13:58.370217 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85047317-e5dc-49b1-9ff3-d424d579ea27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:13:58.392505 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-85047317-e5dc-49b1-9ff3-d424d579ea27 None None] MFA Rules not processed for user `ca155a48e2024c32af3fd2e19e5fac11`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 11:13:58.393729 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-85047317-e5dc-49b1-9ff3-d424d579ea27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:13:58.402254 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1367/2716] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 11:13:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 11:13:58.407263 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c829d9d-e865-44f8-962d-4c8db99876fb None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:13:58.417483 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c829d9d-e865-44f8-962d-4c8db99876fb None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:13:58.424996 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c56449b-0f80-4323-926c-9a492e094c7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:13:58.425693 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c56449b-0f80-4323-926c-9a492e094c7d admin admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:13:58.425990 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c56449b-0f80-4323-926c-9a492e094c7d admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:13:58.426276 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c56449b-0f80-4323-926c-9a492e094c7d admin admin] PATH_INFO: `/v3/domains` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:13:58.426940 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c56449b-0f80-4323-926c-9a492e094c7d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 11:13:58.427244 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c56449b-0f80-4323-926c-9a492e094c7d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:13:58.428884 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c56449b-0f80-4323-926c-9a492e094c7d admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:13:58.440512 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 11:13:58.449013 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1350/2717] 10.222.0.40 () {58 vars in 1153 bytes} [Fri Aug 26 11:13:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 11:13:58.452817 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85047317-e5dc-49b1-9ff3-d424d579ea27 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:13:58.463210 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85047317-e5dc-49b1-9ff3-d424d579ea27 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:13:58.469638 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137f5c7c-2b1c-4efe-a20b-b43d6c401d83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:13:58.470291 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137f5c7c-2b1c-4efe-a20b-b43d6c401d83 admin admin] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:13:58.470563 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137f5c7c-2b1c-4efe-a20b-b43d6c401d83 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:13:58.470830 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137f5c7c-2b1c-4efe-a20b-b43d6c401d83 admin admin] PATH_INFO: `/v3/projects` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:13:58.471476 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-137f5c7c-2b1c-4efe-a20b-b43d6c401d83 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:13:58.473370 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-137f5c7c-2b1c-4efe-a20b-b43d6c401d83 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:13:58.515076 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1368/2718] 10.222.0.40 () {60 vars in 1153 bytes} [Fri Aug 26 11:13:58 2022] POST /identity/v3/projects => generated 365 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 11:13:58.519940 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c56449b-0f80-4323-926c-9a492e094c7d admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:13:58.531851 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c56449b-0f80-4323-926c-9a492e094c7d admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:13:58.603642 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b088974-f7e7-4793-a12c-88435bfea57e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:13:58.604265 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b088974-f7e7-4793-a12c-88435bfea57e admin admin] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:13:58.604543 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b088974-f7e7-4793-a12c-88435bfea57e admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:13:58.604816 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b088974-f7e7-4793-a12c-88435bfea57e admin admin] PATH_INFO: `/v3/users` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:13:58.605447 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b088974-f7e7-4793-a12c-88435bfea57e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:13:58.607434 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b088974-f7e7-4793-a12c-88435bfea57e admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:13:58.643719 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1351/2719] 10.222.0.40 () {60 vars in 1144 bytes} [Fri Aug 26 11:13:58 2022] POST /identity/v3/users => generated 283 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 11:13:58.645018 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137f5c7c-2b1c-4efe-a20b-b43d6c401d83 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:13:58.657028 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137f5c7c-2b1c-4efe-a20b-b43d6c401d83 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:13:58.662608 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b865976-266e-4482-b863-65bbcb72505b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:13:58.663898 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b865976-266e-4482-b863-65bbcb72505b admin admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:13:58.664341 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b865976-266e-4482-b863-65bbcb72505b admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:13:58.664686 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b865976-266e-4482-b863-65bbcb72505b admin admin] PATH_INFO: `/v3/roles` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:13:58.665433 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b865976-266e-4482-b863-65bbcb72505b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:13:58.667148 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b865976-266e-4482-b863-65bbcb72505b admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:13:58.682998 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1369/2720] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 26 11:13:58 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 11:13:58.687832 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b088974-f7e7-4793-a12c-88435bfea57e admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:13:58.698520 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b088974-f7e7-4793-a12c-88435bfea57e admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:13:58.704420 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c869fd26-c66c-4f94-9349-9d5f4e8ceea2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:13:58.705069 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c869fd26-c66c-4f94-9349-9d5f4e8ceea2 admin admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:13:58.705329 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c869fd26-c66c-4f94-9349-9d5f4e8ceea2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:13:58.705602 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c869fd26-c66c-4f94-9349-9d5f4e8ceea2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:13:58.706306 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c869fd26-c66c-4f94-9349-9d5f4e8ceea2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:13:58.707931 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c869fd26-c66c-4f94-9349-9d5f4e8ceea2 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:13:58.720495 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1352/2721] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 26 11:13:58 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 26 11:13:58.724661 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b865976-266e-4482-b863-65bbcb72505b admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:13:58.737186 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b865976-266e-4482-b863-65bbcb72505b admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:13:58.742300 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7acf424f-4ebd-4e25-8f20-b1959484b659 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:13:58.742949 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7acf424f-4ebd-4e25-8f20-b1959484b659 admin admin] REQUEST_METHOD: `PUT` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:13:58.743226 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7acf424f-4ebd-4e25-8f20-b1959484b659 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:13:58.743498 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7acf424f-4ebd-4e25-8f20-b1959484b659 admin admin] PATH_INFO: `/v3/projects/bdcf5155ed73448d821345546b5a9fea/users/da4bdefb25ca491b8ac0314d5d26bb17/roles/23e174ce743b46eabe618c212691da55` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:13:58.761047 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7acf424f-4ebd-4e25-8f20-b1959484b659 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bdcf5155ed73448d821345546b5a9fea, user_id=da4bdefb25ca491b8ac0314d5d26bb17, role_id=23e174ce743b46eabe618c212691da55)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:13:58.763799 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7acf424f-4ebd-4e25-8f20-b1959484b659 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:13:58.777117 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1370/2722] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 26 11:13:58 2022] PUT /identity/v3/projects/bdcf5155ed73448d821345546b5a9fea/users/da4bdefb25ca491b8ac0314d5d26bb17/roles/23e174ce743b46eabe618c212691da55 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 11:13:58.787656 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412eaa3a-6cdd-42c4-8af8-ea0f862d8bab None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:13:58.787962 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412eaa3a-6cdd-42c4-8af8-ea0f862d8bab None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:13:58.788285 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412eaa3a-6cdd-42c4-8af8-ea0f862d8bab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:13:58.818391 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-412eaa3a-6cdd-42c4-8af8-ea0f862d8bab None None] MFA Rules not processed for user `da4bdefb25ca491b8ac0314d5d26bb17`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 11:13:58.819628 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-412eaa3a-6cdd-42c4-8af8-ea0f862d8bab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:13:58.900484 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1353/2723] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 11:13:58 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 11:13:58.913142 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7acf424f-4ebd-4e25-8f20-b1959484b659 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:13:58.915708 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-7acf424f-4ebd-4e25-8f20-b1959484b659 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:13:58.932123 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 11:13:58.979666 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7acf424f-4ebd-4e25-8f20-b1959484b659 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:13:58.984116 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12a9dc9-5305-4336-9ac6-0ebda0b1d9a1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c277338a7f814cc6b01c5419775031bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e30ccf0c1204727b786876c141dbcd2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:13:58.984728 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12a9dc9-5305-4336-9ac6-0ebda0b1d9a1 service cinder] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:13:58.985006 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12a9dc9-5305-4336-9ac6-0ebda0b1d9a1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:13:58.985280 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12a9dc9-5305-4336-9ac6-0ebda0b1d9a1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:13:58.995243 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b12a9dc9-5305-4336-9ac6-0ebda0b1d9a1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:13:58.996693 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b12a9dc9-5305-4336-9ac6-0ebda0b1d9a1 service cinder] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:13:59.007102 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1371/2724] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 11:13:58 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 11:14:01.201468 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30ec11e-4e25-4abe-82cd-fa7b405830c5 None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:14:01.202032 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30ec11e-4e25-4abe-82cd-fa7b405830c5 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:14:01.202032 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30ec11e-4e25-4abe-82cd-fa7b405830c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:14:01.221463 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-f30ec11e-4e25-4abe-82cd-fa7b405830c5 None None] MFA Rules not processed for user `85716f53e88048af9a8fdcbe4415979c`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 11:14:01.222723 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-f30ec11e-4e25-4abe-82cd-fa7b405830c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:14:01.285956 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1354/2725] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 11:14:01 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 11:14:09.002760 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=20 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 11:14:11.278882 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=91,keystone:INSERT=3 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 11:14:36.743554 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12a9dc9-5305-4336-9ac6-0ebda0b1d9a1 service cinder] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:14:36.841219 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12a9dc9-5305-4336-9ac6-0ebda0b1d9a1 service cinder] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:14:36.849070 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5efed1-a63c-48bb-9229-505d860a7f88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:14:36.850168 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5efed1-a63c-48bb-9229-505d860a7f88 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:14:36.850670 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5efed1-a63c-48bb-9229-505d860a7f88 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:14:36.851128 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5efed1-a63c-48bb-9229-505d860a7f88 admin admin] PATH_INFO: `/v3/users/da4bdefb25ca491b8ac0314d5d26bb17` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:14:36.852978 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf5efed1-a63c-48bb-9229-505d860a7f88 admin admin] RBAC: Authorizing `identity:delete_user(user_id=da4bdefb25ca491b8ac0314d5d26bb17)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:14:36.856579 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf5efed1-a63c-48bb-9229-505d860a7f88 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:14:36.921751 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-cf5efed1-a63c-48bb-9229-505d860a7f88 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'da4bdefb25ca491b8ac0314d5d26bb17'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:14:36.922427 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-cf5efed1-a63c-48bb-9229-505d860a7f88 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'da4bdefb25ca491b8ac0314d5d26bb17'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:14:36.930315 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-cf5efed1-a63c-48bb-9229-505d860a7f88 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'da4bdefb25ca491b8ac0314d5d26bb17'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:14:36.930980 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-cf5efed1-a63c-48bb-9229-505d860a7f88 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'da4bdefb25ca491b8ac0314d5d26bb17'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:14:36.942901 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-cf5efed1-a63c-48bb-9229-505d860a7f88 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'da4bdefb25ca491b8ac0314d5d26bb17'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:14:36.955412 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1372/2726] 10.222.0.40 () {58 vars in 1224 bytes} [Fri Aug 26 11:14:36 2022] DELETE /identity/v3/users/da4bdefb25ca491b8ac0314d5d26bb17 => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 11:14:36.960116 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c195529-de73-4dd2-a18f-034083d53e3a None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:14:36.960617 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c195529-de73-4dd2-a18f-034083d53e3a None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:14:36.960719 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c195529-de73-4dd2-a18f-034083d53e3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:14:36.980005 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-7c195529-de73-4dd2-a18f-034083d53e3a None None] MFA Rules not processed for user `ca155a48e2024c32af3fd2e19e5fac11`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 11:14:36.981127 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-7c195529-de73-4dd2-a18f-034083d53e3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:14:37.040160 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1355/2727] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 11:14:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 11:14:37.049904 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5efed1-a63c-48bb-9229-505d860a7f88 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:14:37.052078 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-cf5efed1-a63c-48bb-9229-505d860a7f88 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:14:37.113447 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5efed1-a63c-48bb-9229-505d860a7f88 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:14:37.117393 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc068630-a825-427d-b02a-79add4a21045 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91da2acd2534423091537e1ba068d2b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e30ccf0c1204727b786876c141dbcd2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:14:37.118044 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc068630-a825-427d-b02a-79add4a21045 service neutron] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:14:37.118326 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc068630-a825-427d-b02a-79add4a21045 service neutron] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:14:37.118601 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc068630-a825-427d-b02a-79add4a21045 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:14:37.128135 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc068630-a825-427d-b02a-79add4a21045 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:14:37.129821 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc068630-a825-427d-b02a-79add4a21045 service neutron] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:14:37.139464 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1373/2728] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 11:14:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 11:14:37.432062 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c195529-de73-4dd2-a18f-034083d53e3a None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:14:37.433565 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-7c195529-de73-4dd2-a18f-034083d53e3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:14:37.446231 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c195529-de73-4dd2-a18f-034083d53e3a None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:14:37.451348 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a800b731-bcfe-4ab8-9486-c8f0f104ecd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:14:37.451978 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a800b731-bcfe-4ab8-9486-c8f0f104ecd6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:14:37.452254 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a800b731-bcfe-4ab8-9486-c8f0f104ecd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:14:37.452533 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a800b731-bcfe-4ab8-9486-c8f0f104ecd6 admin admin] PATH_INFO: `/v3/projects/bdcf5155ed73448d821345546b5a9fea` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:14:37.453677 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a800b731-bcfe-4ab8-9486-c8f0f104ecd6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bdcf5155ed73448d821345546b5a9fea)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:14:37.455598 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a800b731-bcfe-4ab8-9486-c8f0f104ecd6 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:14:37.523564 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-a800b731-bcfe-4ab8-9486-c8f0f104ecd6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bdcf5155ed73448d821345546b5a9fea', 'request_id': 'req-a800b731-bcfe-4ab8-9486-c8f0f104ecd6'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:14:37.530154 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-a800b731-bcfe-4ab8-9486-c8f0f104ecd6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bdcf5155ed73448d821345546b5a9fea', 'request_id': 'req-a800b731-bcfe-4ab8-9486-c8f0f104ecd6'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:14:37.536309 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-a800b731-bcfe-4ab8-9486-c8f0f104ecd6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bdcf5155ed73448d821345546b5a9fea', 'request_id': 'req-a800b731-bcfe-4ab8-9486-c8f0f104ecd6'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:14:37.548257 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-a800b731-bcfe-4ab8-9486-c8f0f104ecd6 admin admin] The token cache is being invalidate because project bdcf5155ed73448d821345546b5a9fea was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81423) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 11:14:37.548542 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-a800b731-bcfe-4ab8-9486-c8f0f104ecd6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:14:37.549174 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-a800b731-bcfe-4ab8-9486-c8f0f104ecd6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:14:37.550842 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1356/2729] 10.222.0.40 () {58 vars in 1233 bytes} [Fri Aug 26 11:14:37 2022] DELETE /identity/v3/projects/bdcf5155ed73448d821345546b5a9fea => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 11:14:37.556250 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb71a7a8-3c2f-4e05-af19-099e49cddc76 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:14:37.556536 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb71a7a8-3c2f-4e05-af19-099e49cddc76 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:14:37.556811 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb71a7a8-3c2f-4e05-af19-099e49cddc76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:14:37.576910 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-bb71a7a8-3c2f-4e05-af19-099e49cddc76 None None] MFA Rules not processed for user `ca155a48e2024c32af3fd2e19e5fac11`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 11:14:37.578162 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-bb71a7a8-3c2f-4e05-af19-099e49cddc76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:14:37.595779 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 11:14:37.642916 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1374/2730] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 11:14:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 11:14:37.653118 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57c89ab-edbd-413d-b9e8-b46223e08e92 None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:14:37.653390 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57c89ab-edbd-413d-b9e8-b46223e08e92 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:14:37.653663 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57c89ab-edbd-413d-b9e8-b46223e08e92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:14:37.673960 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-e57c89ab-edbd-413d-b9e8-b46223e08e92 None None] MFA Rules not processed for user `ca155a48e2024c32af3fd2e19e5fac11`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 11:14:37.675233 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-e57c89ab-edbd-413d-b9e8-b46223e08e92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:14:37.683922 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1357/2731] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 11:14:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 11:14:37.688623 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb71a7a8-3c2f-4e05-af19-099e49cddc76 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:14:37.699673 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb71a7a8-3c2f-4e05-af19-099e49cddc76 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:14:37.704397 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6d6b75-7bf0-42eb-a43a-d32094be7e05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:14:37.704997 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6d6b75-7bf0-42eb-a43a-d32094be7e05 admin admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:14:37.705393 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6d6b75-7bf0-42eb-a43a-d32094be7e05 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:14:37.705684 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6d6b75-7bf0-42eb-a43a-d32094be7e05 admin admin] PATH_INFO: `/v3/domains` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:14:37.706501 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af6d6b75-7bf0-42eb-a43a-d32094be7e05 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 11:14:37.707440 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af6d6b75-7bf0-42eb-a43a-d32094be7e05 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:14:37.709596 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af6d6b75-7bf0-42eb-a43a-d32094be7e05 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:14:37.726423 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1375/2732] 10.222.0.40 () {58 vars in 1153 bytes} [Fri Aug 26 11:14:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 11:14:37.731217 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57c89ab-edbd-413d-b9e8-b46223e08e92 None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:14:37.742484 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57c89ab-edbd-413d-b9e8-b46223e08e92 None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:14:37.748472 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79fe6690-4b2b-4877-9730-78d3d8d63010 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:14:37.749322 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fe6690-4b2b-4877-9730-78d3d8d63010 admin admin] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:14:37.749871 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fe6690-4b2b-4877-9730-78d3d8d63010 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:14:37.750306 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fe6690-4b2b-4877-9730-78d3d8d63010 admin admin] PATH_INFO: `/v3/projects` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:14:37.751104 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79fe6690-4b2b-4877-9730-78d3d8d63010 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:14:37.754000 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79fe6690-4b2b-4877-9730-78d3d8d63010 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:14:37.795446 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1358/2733] 10.222.0.40 () {60 vars in 1153 bytes} [Fri Aug 26 11:14:37 2022] POST /identity/v3/projects => generated 375 bytes in 64 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 11:14:37.799646 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6d6b75-7bf0-42eb-a43a-d32094be7e05 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:14:37.810534 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6d6b75-7bf0-42eb-a43a-d32094be7e05 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:14:37.875613 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae40467-ab00-4f34-8c73-061c06a9f280 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:14:37.875613 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae40467-ab00-4f34-8c73-061c06a9f280 admin admin] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:14:37.875613 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae40467-ab00-4f34-8c73-061c06a9f280 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:14:37.875613 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae40467-ab00-4f34-8c73-061c06a9f280 admin admin] PATH_INFO: `/v3/users` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:14:37.876196 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae40467-ab00-4f34-8c73-061c06a9f280 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:14:37.876196 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae40467-ab00-4f34-8c73-061c06a9f280 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:14:37.898734 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1376/2734] 10.222.0.40 () {60 vars in 1144 bytes} [Fri Aug 26 11:14:37 2022] POST /identity/v3/users => generated 288 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 11:14:37.902975 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79fe6690-4b2b-4877-9730-78d3d8d63010 admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:14:37.914307 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79fe6690-4b2b-4877-9730-78d3d8d63010 admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:14:37.919459 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab887ae-4518-45c6-b876-c9aa51a9a8ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:14:37.920068 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab887ae-4518-45c6-b876-c9aa51a9a8ed admin admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:14:37.920346 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab887ae-4518-45c6-b876-c9aa51a9a8ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:14:37.920624 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab887ae-4518-45c6-b876-c9aa51a9a8ed admin admin] PATH_INFO: `/v3/roles` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:14:37.921271 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab887ae-4518-45c6-b876-c9aa51a9a8ed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:14:37.922908 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab887ae-4518-45c6-b876-c9aa51a9a8ed admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:14:37.935112 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1359/2735] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 26 11:14:37 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 11:14:37.939289 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae40467-ab00-4f34-8c73-061c06a9f280 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:14:37.951626 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae40467-ab00-4f34-8c73-061c06a9f280 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:14:37.957451 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d678990-4d57-4868-bc00-2ad2250efc38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:14:37.958304 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d678990-4d57-4868-bc00-2ad2250efc38 admin admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:14:37.958626 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d678990-4d57-4868-bc00-2ad2250efc38 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:14:37.958951 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d678990-4d57-4868-bc00-2ad2250efc38 admin admin] PATH_INFO: `/v3/roles` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:14:37.959679 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d678990-4d57-4868-bc00-2ad2250efc38 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:14:37.961513 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d678990-4d57-4868-bc00-2ad2250efc38 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:14:37.975001 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1377/2736] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 26 11:14:37 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 11:14:37.979657 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab887ae-4518-45c6-b876-c9aa51a9a8ed admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:14:37.993603 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab887ae-4518-45c6-b876-c9aa51a9a8ed admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:14:37.999492 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed58dc3-fada-44f7-8b14-4100c3a71a64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:14:38.000419 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed58dc3-fada-44f7-8b14-4100c3a71a64 admin admin] REQUEST_METHOD: `PUT` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:14:38.000783 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed58dc3-fada-44f7-8b14-4100c3a71a64 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:14:38.000993 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed58dc3-fada-44f7-8b14-4100c3a71a64 admin admin] PATH_INFO: `/v3/projects/5952be90e61b4ecf8e80c3c0c8b559f5/users/531d11ba027d4480acea437d868faa8c/roles/23e174ce743b46eabe618c212691da55` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:14:38.016945 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed58dc3-fada-44f7-8b14-4100c3a71a64 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5952be90e61b4ecf8e80c3c0c8b559f5, user_id=531d11ba027d4480acea437d868faa8c, role_id=23e174ce743b46eabe618c212691da55)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:14:38.020188 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed58dc3-fada-44f7-8b14-4100c3a71a64 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:14:38.033677 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1360/2737] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 26 11:14:37 2022] PUT /identity/v3/projects/5952be90e61b4ecf8e80c3c0c8b559f5/users/531d11ba027d4480acea437d868faa8c/roles/23e174ce743b46eabe618c212691da55 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 11:14:38.047232 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723b29d6-82d0-4110-8e71-0ef72d76129b None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:14:38.047439 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723b29d6-82d0-4110-8e71-0ef72d76129b None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:14:38.047736 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723b29d6-82d0-4110-8e71-0ef72d76129b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:14:38.083591 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-723b29d6-82d0-4110-8e71-0ef72d76129b None None] MFA Rules not processed for user `531d11ba027d4480acea437d868faa8c`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 11:14:38.085102 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-723b29d6-82d0-4110-8e71-0ef72d76129b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:14:38.165875 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1378/2738] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 11:14:38 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 11:14:38.178845 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed58dc3-fada-44f7-8b14-4100c3a71a64 admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:14:38.180884 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-4ed58dc3-fada-44f7-8b14-4100c3a71a64 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:14:38.197996 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 11:14:38.244877 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed58dc3-fada-44f7-8b14-4100c3a71a64 admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:14:38.248586 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f4cda8-caac-4251-8fc5-2566a9ed43d7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c277338a7f814cc6b01c5419775031bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e30ccf0c1204727b786876c141dbcd2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:14:38.249193 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f4cda8-caac-4251-8fc5-2566a9ed43d7 service cinder] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:14:38.249469 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f4cda8-caac-4251-8fc5-2566a9ed43d7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:14:38.250058 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f4cda8-caac-4251-8fc5-2566a9ed43d7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:14:38.259929 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f4cda8-caac-4251-8fc5-2566a9ed43d7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:14:38.261400 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f4cda8-caac-4251-8fc5-2566a9ed43d7 service cinder] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:14:38.271410 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1361/2739] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 11:14:38 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 11:14:48.161590 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=89,keystone:INSERT=3 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 11:14:48.266475 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=21 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 11:14:48.299394 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723b29d6-82d0-4110-8e71-0ef72d76129b None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:14:48.367642 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723b29d6-82d0-4110-8e71-0ef72d76129b None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:14:48.372260 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d75b48-ea31-4b6b-86c5-e151b65f570f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:14:48.372894 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d75b48-ea31-4b6b-86c5-e151b65f570f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:14:48.373172 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d75b48-ea31-4b6b-86c5-e151b65f570f admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:14:48.373444 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d75b48-ea31-4b6b-86c5-e151b65f570f admin admin] PATH_INFO: `/v3/users/531d11ba027d4480acea437d868faa8c` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:14:48.374600 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d75b48-ea31-4b6b-86c5-e151b65f570f admin admin] RBAC: Authorizing `identity:delete_user(user_id=531d11ba027d4480acea437d868faa8c)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:14:48.376796 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d75b48-ea31-4b6b-86c5-e151b65f570f admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:14:48.430189 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-10d75b48-ea31-4b6b-86c5-e151b65f570f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '531d11ba027d4480acea437d868faa8c'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:14:48.430848 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-10d75b48-ea31-4b6b-86c5-e151b65f570f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '531d11ba027d4480acea437d868faa8c'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:14:48.438387 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-10d75b48-ea31-4b6b-86c5-e151b65f570f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '531d11ba027d4480acea437d868faa8c'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:14:48.439089 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-10d75b48-ea31-4b6b-86c5-e151b65f570f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '531d11ba027d4480acea437d868faa8c'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:14:48.451487 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-10d75b48-ea31-4b6b-86c5-e151b65f570f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '531d11ba027d4480acea437d868faa8c'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:14:48.465346 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1379/2740] 10.222.0.40 () {58 vars in 1224 bytes} [Fri Aug 26 11:14:48 2022] DELETE /identity/v3/users/531d11ba027d4480acea437d868faa8c => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 11:14:48.469908 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa425c7-fba9-482e-9c96-44fd184976bc None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:14:48.470247 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa425c7-fba9-482e-9c96-44fd184976bc None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:14:48.470530 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa425c7-fba9-482e-9c96-44fd184976bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:14:48.490054 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-ffa425c7-fba9-482e-9c96-44fd184976bc None None] MFA Rules not processed for user `ca155a48e2024c32af3fd2e19e5fac11`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 11:14:48.491293 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-ffa425c7-fba9-482e-9c96-44fd184976bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:14:48.552583 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1362/2741] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 11:14:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 11:14:48.562356 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d75b48-ea31-4b6b-86c5-e151b65f570f admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:14:48.563793 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-10d75b48-ea31-4b6b-86c5-e151b65f570f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:14:48.626931 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d75b48-ea31-4b6b-86c5-e151b65f570f admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:14:48.631045 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-febe40fa-b23b-4690-9bb5-a8934a8f6c7b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91da2acd2534423091537e1ba068d2b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e30ccf0c1204727b786876c141dbcd2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:14:48.631660 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febe40fa-b23b-4690-9bb5-a8934a8f6c7b service neutron] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:14:48.631940 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febe40fa-b23b-4690-9bb5-a8934a8f6c7b service neutron] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:14:48.632214 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febe40fa-b23b-4690-9bb5-a8934a8f6c7b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:14:48.641921 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-febe40fa-b23b-4690-9bb5-a8934a8f6c7b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:14:48.643397 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-febe40fa-b23b-4690-9bb5-a8934a8f6c7b service neutron] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:14:48.653902 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1380/2742] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 11:14:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 11:14:48.932699 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa425c7-fba9-482e-9c96-44fd184976bc None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:14:48.934269 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-ffa425c7-fba9-482e-9c96-44fd184976bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:14:48.946855 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa425c7-fba9-482e-9c96-44fd184976bc None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:14:48.951552 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93579450-4a7b-42ed-8c20-577a6b66790f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:14:48.952194 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93579450-4a7b-42ed-8c20-577a6b66790f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:14:48.952539 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93579450-4a7b-42ed-8c20-577a6b66790f admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:14:48.952737 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93579450-4a7b-42ed-8c20-577a6b66790f admin admin] PATH_INFO: `/v3/projects/5952be90e61b4ecf8e80c3c0c8b559f5` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:14:48.953876 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93579450-4a7b-42ed-8c20-577a6b66790f admin admin] RBAC: Authorizing `identity:delete_project(project_id=5952be90e61b4ecf8e80c3c0c8b559f5)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:14:48.955836 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93579450-4a7b-42ed-8c20-577a6b66790f admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:14:49.007724 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-93579450-4a7b-42ed-8c20-577a6b66790f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5952be90e61b4ecf8e80c3c0c8b559f5', 'request_id': 'req-93579450-4a7b-42ed-8c20-577a6b66790f'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:14:49.014043 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-93579450-4a7b-42ed-8c20-577a6b66790f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5952be90e61b4ecf8e80c3c0c8b559f5', 'request_id': 'req-93579450-4a7b-42ed-8c20-577a6b66790f'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:14:49.020605 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-93579450-4a7b-42ed-8c20-577a6b66790f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5952be90e61b4ecf8e80c3c0c8b559f5', 'request_id': 'req-93579450-4a7b-42ed-8c20-577a6b66790f'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:14:49.035241 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-93579450-4a7b-42ed-8c20-577a6b66790f admin admin] The token cache is being invalidate because project 5952be90e61b4ecf8e80c3c0c8b559f5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81423) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 11:14:49.035669 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-93579450-4a7b-42ed-8c20-577a6b66790f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:14:49.036397 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-93579450-4a7b-42ed-8c20-577a6b66790f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:14:49.038069 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1363/2743] 10.222.0.40 () {58 vars in 1233 bytes} [Fri Aug 26 11:14:48 2022] DELETE /identity/v3/projects/5952be90e61b4ecf8e80c3c0c8b559f5 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 26 11:14:49.043020 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f482060-eaa8-4444-a826-4ba835b61da5 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:14:49.043302 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f482060-eaa8-4444-a826-4ba835b61da5 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:14:49.043622 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f482060-eaa8-4444-a826-4ba835b61da5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:14:49.064540 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-9f482060-eaa8-4444-a826-4ba835b61da5 None None] MFA Rules not processed for user `ca155a48e2024c32af3fd2e19e5fac11`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 11:14:49.065911 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-9f482060-eaa8-4444-a826-4ba835b61da5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:14:49.084281 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 11:14:49.137357 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1381/2744] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 11:14:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 11:14:49.147938 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8300eb20-82f1-46bc-b640-79f8fe09ff1f None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:14:49.148219 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8300eb20-82f1-46bc-b640-79f8fe09ff1f None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:14:49.148491 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8300eb20-82f1-46bc-b640-79f8fe09ff1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:14:49.169265 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-8300eb20-82f1-46bc-b640-79f8fe09ff1f None None] MFA Rules not processed for user `ca155a48e2024c32af3fd2e19e5fac11`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 11:14:49.170526 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-8300eb20-82f1-46bc-b640-79f8fe09ff1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:14:49.178691 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1364/2745] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 11:14:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 11:14:49.183025 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f482060-eaa8-4444-a826-4ba835b61da5 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:14:49.193248 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f482060-eaa8-4444-a826-4ba835b61da5 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:14:49.198104 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49ede76-9cc6-45f5-a660-8ece7bf948e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:14:49.198723 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49ede76-9cc6-45f5-a660-8ece7bf948e6 admin admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:14:49.199001 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49ede76-9cc6-45f5-a660-8ece7bf948e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:14:49.199274 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49ede76-9cc6-45f5-a660-8ece7bf948e6 admin admin] PATH_INFO: `/v3/domains` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:14:49.199927 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f49ede76-9cc6-45f5-a660-8ece7bf948e6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 11:14:49.200229 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f49ede76-9cc6-45f5-a660-8ece7bf948e6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:14:49.201829 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f49ede76-9cc6-45f5-a660-8ece7bf948e6 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:14:49.214753 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1382/2746] 10.222.0.40 () {58 vars in 1153 bytes} [Fri Aug 26 11:14:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 11:14:49.219208 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8300eb20-82f1-46bc-b640-79f8fe09ff1f None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:14:49.230570 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8300eb20-82f1-46bc-b640-79f8fe09ff1f None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:14:49.235290 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9d7054-8463-4535-8cb3-74b10247ec25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:14:49.235890 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9d7054-8463-4535-8cb3-74b10247ec25 admin admin] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:14:49.236167 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9d7054-8463-4535-8cb3-74b10247ec25 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:14:49.236436 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9d7054-8463-4535-8cb3-74b10247ec25 admin admin] PATH_INFO: `/v3/projects` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:14:49.239428 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c9d7054-8463-4535-8cb3-74b10247ec25 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:14:49.239428 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c9d7054-8463-4535-8cb3-74b10247ec25 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:14:49.271107 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1365/2747] 10.222.0.40 () {60 vars in 1153 bytes} [Fri Aug 26 11:14:49 2022] POST /identity/v3/projects => generated 373 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 11:14:49.275769 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49ede76-9cc6-45f5-a660-8ece7bf948e6 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:14:49.287099 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49ede76-9cc6-45f5-a660-8ece7bf948e6 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:14:49.346797 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540bea23-e2a6-4fc6-b527-5b740d86d9e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:14:49.347995 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540bea23-e2a6-4fc6-b527-5b740d86d9e7 admin admin] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:14:49.348360 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540bea23-e2a6-4fc6-b527-5b740d86d9e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:14:49.348784 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540bea23-e2a6-4fc6-b527-5b740d86d9e7 admin admin] PATH_INFO: `/v3/users` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:14:49.349423 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-540bea23-e2a6-4fc6-b527-5b740d86d9e7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:14:49.351532 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-540bea23-e2a6-4fc6-b527-5b740d86d9e7 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:14:49.375831 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1383/2748] 10.222.0.40 () {60 vars in 1144 bytes} [Fri Aug 26 11:14:49 2022] POST /identity/v3/users => generated 287 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 11:14:49.379746 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9d7054-8463-4535-8cb3-74b10247ec25 admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:14:49.390587 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9d7054-8463-4535-8cb3-74b10247ec25 admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:14:49.396079 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b02d7e-3a44-4d00-9180-8813488086a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:14:49.396692 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b02d7e-3a44-4d00-9180-8813488086a3 admin admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:14:49.396972 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b02d7e-3a44-4d00-9180-8813488086a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:14:49.397243 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b02d7e-3a44-4d00-9180-8813488086a3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:14:49.397893 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b02d7e-3a44-4d00-9180-8813488086a3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:14:49.399541 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b02d7e-3a44-4d00-9180-8813488086a3 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:14:49.411713 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1366/2749] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 26 11:14:49 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 11:14:49.415556 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540bea23-e2a6-4fc6-b527-5b740d86d9e7 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:14:49.426548 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540bea23-e2a6-4fc6-b527-5b740d86d9e7 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:14:49.431512 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862f4295-e28e-4f1b-9fd0-4d9e0950807a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:14:49.432116 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862f4295-e28e-4f1b-9fd0-4d9e0950807a admin admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:14:49.432395 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862f4295-e28e-4f1b-9fd0-4d9e0950807a admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:14:49.432663 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862f4295-e28e-4f1b-9fd0-4d9e0950807a admin admin] PATH_INFO: `/v3/roles` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:14:49.433300 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-862f4295-e28e-4f1b-9fd0-4d9e0950807a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:14:49.434929 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-862f4295-e28e-4f1b-9fd0-4d9e0950807a admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:14:49.446816 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1384/2750] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 26 11:14:49 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 11:14:49.450999 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b02d7e-3a44-4d00-9180-8813488086a3 admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:14:49.461862 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b02d7e-3a44-4d00-9180-8813488086a3 admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:14:49.466530 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d490330-ad8a-4a37-8266-b5eb809c39bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:14:49.467181 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d490330-ad8a-4a37-8266-b5eb809c39bd admin admin] REQUEST_METHOD: `PUT` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:14:49.467453 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d490330-ad8a-4a37-8266-b5eb809c39bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:14:49.468359 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d490330-ad8a-4a37-8266-b5eb809c39bd admin admin] PATH_INFO: `/v3/projects/3d84157876f945099fda5fcb9d3f6c20/users/93bec07110aa4697ab59ced454a51f9a/roles/23e174ce743b46eabe618c212691da55` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:14:49.486732 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d490330-ad8a-4a37-8266-b5eb809c39bd admin admin] RBAC: Authorizing `identity:create_grant(project_id=3d84157876f945099fda5fcb9d3f6c20, user_id=93bec07110aa4697ab59ced454a51f9a, role_id=23e174ce743b46eabe618c212691da55)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:14:49.489429 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d490330-ad8a-4a37-8266-b5eb809c39bd admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:14:49.502704 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1367/2751] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 26 11:14:49 2022] PUT /identity/v3/projects/3d84157876f945099fda5fcb9d3f6c20/users/93bec07110aa4697ab59ced454a51f9a/roles/23e174ce743b46eabe618c212691da55 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 26 11:14:49.508943 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc503c4-2123-439e-a0e0-5c033ae7cac7 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:14:49.509367 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc503c4-2123-439e-a0e0-5c033ae7cac7 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:14:49.509747 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc503c4-2123-439e-a0e0-5c033ae7cac7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:14:49.529687 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-edc503c4-2123-439e-a0e0-5c033ae7cac7 None None] MFA Rules not processed for user `ca155a48e2024c32af3fd2e19e5fac11`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 11:14:49.531017 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-edc503c4-2123-439e-a0e0-5c033ae7cac7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:14:49.592188 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1385/2752] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 11:14:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 11:14:49.602618 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d490330-ad8a-4a37-8266-b5eb809c39bd admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:14:49.604257 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-0d490330-ad8a-4a37-8266-b5eb809c39bd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:14:49.626900 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 11:14:49.667778 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d490330-ad8a-4a37-8266-b5eb809c39bd admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:14:49.671890 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3d0ca4-92c7-4365-bf23-db5c1109ba08 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91da2acd2534423091537e1ba068d2b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e30ccf0c1204727b786876c141dbcd2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:14:49.672504 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3d0ca4-92c7-4365-bf23-db5c1109ba08 service neutron] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:14:49.672783 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3d0ca4-92c7-4365-bf23-db5c1109ba08 service neutron] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:14:49.673056 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3d0ca4-92c7-4365-bf23-db5c1109ba08 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:14:49.684377 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d3d0ca4-92c7-4365-bf23-db5c1109ba08 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:14:49.685819 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d3d0ca4-92c7-4365-bf23-db5c1109ba08 service neutron] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:14:49.696811 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1368/2753] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 11:14:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 11:14:53.642942 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b81f06f-3633-47cf-885a-65e4061d94ca None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:14:53.643527 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b81f06f-3633-47cf-885a-65e4061d94ca None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:14:53.643527 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b81f06f-3633-47cf-885a-65e4061d94ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:14:53.686371 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-2b81f06f-3633-47cf-885a-65e4061d94ca None None] MFA Rules not processed for user `93bec07110aa4697ab59ced454a51f9a`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 11:14:53.689009 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-2b81f06f-3633-47cf-885a-65e4061d94ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:14:53.719317 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 11:14:53.790393 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1386/2754] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 11:14:53 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 11:14:53.794782 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3d0ca4-92c7-4365-bf23-db5c1109ba08 service neutron] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:14:53.807224 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3d0ca4-92c7-4365-bf23-db5c1109ba08 service neutron] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:14:53.812404 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f93f95-77c4-4482-82f2-c339d28b2734 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:14:53.818017 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f93f95-77c4-4482-82f2-c339d28b2734 admin admin] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:14:53.818017 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f93f95-77c4-4482-82f2-c339d28b2734 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:14:53.818017 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f93f95-77c4-4482-82f2-c339d28b2734 admin admin] PATH_INFO: `/v3/projects` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:14:53.818017 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f93f95-77c4-4482-82f2-c339d28b2734 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:14:53.818017 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f93f95-77c4-4482-82f2-c339d28b2734 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:14:53.846275 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1369/2755] 10.222.0.40 () {60 vars in 1153 bytes} [Fri Aug 26 11:14:53 2022] POST /identity/v3/projects => generated 373 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 11:14:53.850798 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b81f06f-3633-47cf-885a-65e4061d94ca None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:14:53.864259 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b81f06f-3633-47cf-885a-65e4061d94ca None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:14:53.932831 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7097f3-3340-4520-806d-34997b9154a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:14:53.933494 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7097f3-3340-4520-806d-34997b9154a8 admin admin] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:14:53.933767 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7097f3-3340-4520-806d-34997b9154a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:14:53.934072 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7097f3-3340-4520-806d-34997b9154a8 admin admin] PATH_INFO: `/v3/users` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:14:53.934728 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c7097f3-3340-4520-806d-34997b9154a8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:14:53.936651 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c7097f3-3340-4520-806d-34997b9154a8 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:14:53.961558 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1387/2756] 10.222.0.40 () {60 vars in 1144 bytes} [Fri Aug 26 11:14:53 2022] POST /identity/v3/users => generated 293 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 11:14:53.965573 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f93f95-77c4-4482-82f2-c339d28b2734 admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:14:53.977376 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f93f95-77c4-4482-82f2-c339d28b2734 admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:14:53.982210 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fbdbad9-3220-4f30-872d-6894744ae1d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:14:53.982851 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbdbad9-3220-4f30-872d-6894744ae1d1 admin admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:14:53.983079 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbdbad9-3220-4f30-872d-6894744ae1d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:14:53.983345 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbdbad9-3220-4f30-872d-6894744ae1d1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:14:53.984075 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fbdbad9-3220-4f30-872d-6894744ae1d1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:14:53.985626 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fbdbad9-3220-4f30-872d-6894744ae1d1 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:14:54.012982 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1370/2757] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 26 11:14:53 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 11:14:54.019804 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7097f3-3340-4520-806d-34997b9154a8 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:14:54.036396 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7097f3-3340-4520-806d-34997b9154a8 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:14:54.043580 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b4f564-e3f4-4c11-bd59-69608b08adf9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:14:54.044275 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b4f564-e3f4-4c11-bd59-69608b08adf9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:14:54.044554 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b4f564-e3f4-4c11-bd59-69608b08adf9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:14:54.044835 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b4f564-e3f4-4c11-bd59-69608b08adf9 admin admin] PATH_INFO: `/v3/projects/3b7d623925884b588481227e89255494/users/c5f437d3ad9847069fda5361c153cd6c/roles/1f97e4602e2e4d6498913a81f9ae8d6a` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:14:54.064221 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b4f564-e3f4-4c11-bd59-69608b08adf9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3b7d623925884b588481227e89255494, user_id=c5f437d3ad9847069fda5361c153cd6c, role_id=1f97e4602e2e4d6498913a81f9ae8d6a)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:14:54.067114 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b4f564-e3f4-4c11-bd59-69608b08adf9 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:14:54.082807 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1388/2758] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 26 11:14:54 2022] PUT /identity/v3/projects/3b7d623925884b588481227e89255494/users/c5f437d3ad9847069fda5361c153cd6c/roles/1f97e4602e2e4d6498913a81f9ae8d6a => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 26 11:14:54.086935 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fbdbad9-3220-4f30-872d-6894744ae1d1 admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:14:54.101131 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fbdbad9-3220-4f30-872d-6894744ae1d1 admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:14:54.178878 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6f9538-685a-4da1-b9d8-c6343ab7c7ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:14:54.179815 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6f9538-685a-4da1-b9d8-c6343ab7c7ee admin admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:14:54.180034 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6f9538-685a-4da1-b9d8-c6343ab7c7ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:14:54.180367 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6f9538-685a-4da1-b9d8-c6343ab7c7ee admin admin] PATH_INFO: `/v3/roles` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:14:54.181100 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a6f9538-685a-4da1-b9d8-c6343ab7c7ee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:14:54.182756 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a6f9538-685a-4da1-b9d8-c6343ab7c7ee admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:14:54.198990 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1371/2759] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 26 11:14:54 2022] GET /identity/v3/roles => generated 1420 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 11:14:54.200249 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b4f564-e3f4-4c11-bd59-69608b08adf9 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:14:54.213026 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b4f564-e3f4-4c11-bd59-69608b08adf9 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:14:54.219020 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09da0bc1-9e7c-49f9-9ed8-23de20846936 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:14:54.219811 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09da0bc1-9e7c-49f9-9ed8-23de20846936 admin admin] REQUEST_METHOD: `PUT` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:14:54.220285 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09da0bc1-9e7c-49f9-9ed8-23de20846936 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:14:54.220686 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09da0bc1-9e7c-49f9-9ed8-23de20846936 admin admin] PATH_INFO: `/v3/projects/3b7d623925884b588481227e89255494/users/c5f437d3ad9847069fda5361c153cd6c/roles/1f97e4602e2e4d6498913a81f9ae8d6a` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:14:54.224057 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09da0bc1-9e7c-49f9-9ed8-23de20846936 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3b7d623925884b588481227e89255494, user_id=c5f437d3ad9847069fda5361c153cd6c, role_id=1f97e4602e2e4d6498913a81f9ae8d6a)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:14:54.227038 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09da0bc1-9e7c-49f9-9ed8-23de20846936 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:14:54.240937 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1389/2760] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 26 11:14:54 2022] PUT /identity/v3/projects/3b7d623925884b588481227e89255494/users/c5f437d3ad9847069fda5361c153cd6c/roles/1f97e4602e2e4d6498913a81f9ae8d6a => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 11:14:59.454614 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442be1ad-0a0e-4b80-8ab9-77422ccf3de9 None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:14:59.455212 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442be1ad-0a0e-4b80-8ab9-77422ccf3de9 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:14:59.455212 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442be1ad-0a0e-4b80-8ab9-77422ccf3de9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:14:59.493317 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-442be1ad-0a0e-4b80-8ab9-77422ccf3de9 None None] MFA Rules not processed for user `c5f437d3ad9847069fda5361c153cd6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 11:14:59.494734 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-442be1ad-0a0e-4b80-8ab9-77422ccf3de9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:14:59.578033 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1372/2761] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 11:14:59 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 11:14:59.597724 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09da0bc1-9e7c-49f9-9ed8-23de20846936 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:14:59.600232 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-09da0bc1-9e7c-49f9-9ed8-23de20846936 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:14:59.689657 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 11:14:59.723207 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09da0bc1-9e7c-49f9-9ed8-23de20846936 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:14:59.731055 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43d15ed-fa4a-4dba-a235-7620b3045532 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f4b4a662db049ecacc865ecff10bdb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e30ccf0c1204727b786876c141dbcd2', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:14:59.731844 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43d15ed-fa4a-4dba-a235-7620b3045532 service nova] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:14:59.732119 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43d15ed-fa4a-4dba-a235-7620b3045532 service nova] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:14:59.732385 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43d15ed-fa4a-4dba-a235-7620b3045532 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:14:59.799754 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e43d15ed-fa4a-4dba-a235-7620b3045532 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:14:59.802358 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e43d15ed-fa4a-4dba-a235-7620b3045532 service nova] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:14:59.812638 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1390/2762] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 11:14:59 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 216 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 11:15:00.556405 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442be1ad-0a0e-4b80-8ab9-77422ccf3de9 None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:15:00.558926 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-442be1ad-0a0e-4b80-8ab9-77422ccf3de9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:15:00.583465 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 11:15:00.661509 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442be1ad-0a0e-4b80-8ab9-77422ccf3de9 None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:15:00.665276 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74040375-b4d6-4f7f-a67c-bba9405ac5d2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c277338a7f814cc6b01c5419775031bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e30ccf0c1204727b786876c141dbcd2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:15:00.665894 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74040375-b4d6-4f7f-a67c-bba9405ac5d2 service cinder] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:15:00.666208 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74040375-b4d6-4f7f-a67c-bba9405ac5d2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:15:00.666532 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74040375-b4d6-4f7f-a67c-bba9405ac5d2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:15:00.668385 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-74040375-b4d6-4f7f-a67c-bba9405ac5d2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:15:00.684175 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74040375-b4d6-4f7f-a67c-bba9405ac5d2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:15:00.685640 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74040375-b4d6-4f7f-a67c-bba9405ac5d2 service cinder] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:15:00.698585 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1373/2763] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 11:15:00 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 11:15:00.974841 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b330c0-e667-4957-ae35-1ee38338aa38 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:15:00.975495 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b330c0-e667-4957-ae35-1ee38338aa38 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:15:00.975495 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b330c0-e667-4957-ae35-1ee38338aa38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:15:01.001005 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-a0b330c0-e667-4957-ae35-1ee38338aa38 None None] MFA Rules not processed for user `93bec07110aa4697ab59ced454a51f9a`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 11:15:01.002399 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-a0b330c0-e667-4957-ae35-1ee38338aa38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:15:01.010365 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1391/2764] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 11:15:00 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 11:15:08.743463 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74040375-b4d6-4f7f-a67c-bba9405ac5d2 service cinder] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:15:08.752394 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74040375-b4d6-4f7f-a67c-bba9405ac5d2 service cinder] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:15:08.757151 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28546f0a-5852-41f9-8646-0adca915e28c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c277338a7f814cc6b01c5419775031bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e30ccf0c1204727b786876c141dbcd2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:15:08.757843 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28546f0a-5852-41f9-8646-0adca915e28c service cinder] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:15:08.758319 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28546f0a-5852-41f9-8646-0adca915e28c service cinder] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:15:08.758699 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28546f0a-5852-41f9-8646-0adca915e28c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:15:08.760659 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-28546f0a-5852-41f9-8646-0adca915e28c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:15:08.777053 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28546f0a-5852-41f9-8646-0adca915e28c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:15:08.778561 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28546f0a-5852-41f9-8646-0adca915e28c service cinder] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:15:08.791359 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1374/2765] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 11:15:08 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 11:15:10.076522 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b330c0-e667-4957-ae35-1ee38338aa38 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:15:10.078073 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-a0b330c0-e667-4957-ae35-1ee38338aa38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:15:10.092342 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b330c0-e667-4957-ae35-1ee38338aa38 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:15:10.095908 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6d8ec4-95ab-406f-aab0-9fbc79d52c46 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c277338a7f814cc6b01c5419775031bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e30ccf0c1204727b786876c141dbcd2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:15:10.096516 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6d8ec4-95ab-406f-aab0-9fbc79d52c46 service cinder] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:15:10.098194 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6d8ec4-95ab-406f-aab0-9fbc79d52c46 service cinder] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:15:10.098502 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6d8ec4-95ab-406f-aab0-9fbc79d52c46 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:15:10.100361 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-5c6d8ec4-95ab-406f-aab0-9fbc79d52c46 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:15:10.164723 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c6d8ec4-95ab-406f-aab0-9fbc79d52c46 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:15:10.166212 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c6d8ec4-95ab-406f-aab0-9fbc79d52c46 service cinder] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:15:10.176736 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1392/2766] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 11:15:10 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 11:15:16.288763 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28546f0a-5852-41f9-8646-0adca915e28c service cinder] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:15:16.297503 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28546f0a-5852-41f9-8646-0adca915e28c service cinder] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:15:16.302405 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0954bf3-0159-42ca-b12b-75044dfbee20 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c277338a7f814cc6b01c5419775031bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e30ccf0c1204727b786876c141dbcd2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:15:16.303021 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0954bf3-0159-42ca-b12b-75044dfbee20 service cinder] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:15:16.303337 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0954bf3-0159-42ca-b12b-75044dfbee20 service cinder] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:15:16.303634 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0954bf3-0159-42ca-b12b-75044dfbee20 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:15:16.305457 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-e0954bf3-0159-42ca-b12b-75044dfbee20 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:15:16.384059 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0954bf3-0159-42ca-b12b-75044dfbee20 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:15:16.385616 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0954bf3-0159-42ca-b12b-75044dfbee20 service cinder] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:15:16.397022 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1375/2767] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 11:15:16 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 11:15:17.198437 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6d8ec4-95ab-406f-aab0-9fbc79d52c46 service cinder] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:15:17.200587 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-5c6d8ec4-95ab-406f-aab0-9fbc79d52c46 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:15:17.285240 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6d8ec4-95ab-406f-aab0-9fbc79d52c46 service cinder] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:15:17.290673 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78423b04-2b37-477d-9aac-efc150f1962c service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f29c5dfed31f42a9b5cbc0cd72e9fdc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e30ccf0c1204727b786876c141dbcd2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:15:17.291321 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78423b04-2b37-477d-9aac-efc150f1962c service placement] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:15:17.291630 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78423b04-2b37-477d-9aac-efc150f1962c service placement] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:15:17.291892 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78423b04-2b37-477d-9aac-efc150f1962c service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:15:17.293609 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-78423b04-2b37-477d-9aac-efc150f1962c service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:15:17.306462 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78423b04-2b37-477d-9aac-efc150f1962c service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:15:17.308040 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78423b04-2b37-477d-9aac-efc150f1962c service placement] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:15:17.318995 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1393/2768] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 11:15:17 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 11:15:20.524794 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0954bf3-0159-42ca-b12b-75044dfbee20 service cinder] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:15:20.526824 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-e0954bf3-0159-42ca-b12b-75044dfbee20 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:15:20.597435 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0954bf3-0159-42ca-b12b-75044dfbee20 service cinder] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:15:20.601615 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5eeb02-8d44-4434-a10d-8237c88158df service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91da2acd2534423091537e1ba068d2b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e30ccf0c1204727b786876c141dbcd2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:15:20.602343 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5eeb02-8d44-4434-a10d-8237c88158df service neutron] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:15:20.602671 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5eeb02-8d44-4434-a10d-8237c88158df service neutron] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:15:20.603159 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5eeb02-8d44-4434-a10d-8237c88158df service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:15:20.605139 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-6d5eeb02-8d44-4434-a10d-8237c88158df service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:15:20.665221 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 11:15:20.679717 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d5eeb02-8d44-4434-a10d-8237c88158df service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:15:20.681335 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d5eeb02-8d44-4434-a10d-8237c88158df service neutron] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:15:20.692662 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1376/2769] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 11:15:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 11:15:21.072009 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78423b04-2b37-477d-9aac-efc150f1962c service placement] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:15:21.085859 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78423b04-2b37-477d-9aac-efc150f1962c service placement] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:15:21.092058 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2b1685-c673-4c04-aea7-ea7fddc90bfd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91da2acd2534423091537e1ba068d2b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e30ccf0c1204727b786876c141dbcd2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:15:21.093219 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2b1685-c673-4c04-aea7-ea7fddc90bfd service neutron] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:15:21.093503 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2b1685-c673-4c04-aea7-ea7fddc90bfd service neutron] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:15:21.093888 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2b1685-c673-4c04-aea7-ea7fddc90bfd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:15:21.096309 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-fa2b1685-c673-4c04-aea7-ea7fddc90bfd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:15:21.104443 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 11:15:21.107869 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa2b1685-c673-4c04-aea7-ea7fddc90bfd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:15:21.109784 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa2b1685-c673-4c04-aea7-ea7fddc90bfd service neutron] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:15:21.140571 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1394/2770] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 11:15:21 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 69 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 11:15:24.562245 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4a43f2-32de-4e00-a1c6-75aaf2b06076 None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:15:24.564090 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4a43f2-32de-4e00-a1c6-75aaf2b06076 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:15:24.564575 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4a43f2-32de-4e00-a1c6-75aaf2b06076 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:15:24.590929 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-7e4a43f2-32de-4e00-a1c6-75aaf2b06076 None None] MFA Rules not processed for user `9e8355dc324e49d881b9f0fc5c2923ad`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 11:15:24.592651 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-7e4a43f2-32de-4e00-a1c6-75aaf2b06076 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:15:24.603924 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1377/2771] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 11:15:24 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 11:15:25.072988 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2b1685-c673-4c04-aea7-ea7fddc90bfd service neutron] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:15:25.077254 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-fa2b1685-c673-4c04-aea7-ea7fddc90bfd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:15:25.109883 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2b1685-c673-4c04-aea7-ea7fddc90bfd service neutron] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:15:25.117388 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca360e6b-3901-4085-bf7c-803538bf76af service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f29c5dfed31f42a9b5cbc0cd72e9fdc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e30ccf0c1204727b786876c141dbcd2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:15:25.118129 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca360e6b-3901-4085-bf7c-803538bf76af service placement] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:15:25.118706 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca360e6b-3901-4085-bf7c-803538bf76af service placement] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:15:25.118996 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca360e6b-3901-4085-bf7c-803538bf76af service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:15:25.121955 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-ca360e6b-3901-4085-bf7c-803538bf76af service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:15:25.138782 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca360e6b-3901-4085-bf7c-803538bf76af service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:15:25.140812 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca360e6b-3901-4085-bf7c-803538bf76af service placement] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:15:25.156134 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4a43f2-32de-4e00-a1c6-75aaf2b06076 None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:15:25.157687 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-7e4a43f2-32de-4e00-a1c6-75aaf2b06076 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:15:25.159703 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1395/2772] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 11:15:25 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 11:15:25.182824 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4a43f2-32de-4e00-a1c6-75aaf2b06076 None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:15:25.189143 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd7071c-bd44-47f2-a699-6c9ae27b9f56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:15:25.189925 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd7071c-bd44-47f2-a699-6c9ae27b9f56 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:15:25.190995 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd7071c-bd44-47f2-a699-6c9ae27b9f56 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:15:25.191517 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd7071c-bd44-47f2-a699-6c9ae27b9f56 admin admin] PATH_INFO: `/v3/users/85716f53e88048af9a8fdcbe4415979c` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:15:25.193173 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd7071c-bd44-47f2-a699-6c9ae27b9f56 admin admin] RBAC: Authorizing `identity:delete_user(user_id=85716f53e88048af9a8fdcbe4415979c)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:15:25.196004 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd7071c-bd44-47f2-a699-6c9ae27b9f56 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:15:25.250705 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-ddd7071c-bd44-47f2-a699-6c9ae27b9f56 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '85716f53e88048af9a8fdcbe4415979c'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:15:25.259192 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-ddd7071c-bd44-47f2-a699-6c9ae27b9f56 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '85716f53e88048af9a8fdcbe4415979c'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:15:25.260008 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-ddd7071c-bd44-47f2-a699-6c9ae27b9f56 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '85716f53e88048af9a8fdcbe4415979c'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:15:25.277740 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-ddd7071c-bd44-47f2-a699-6c9ae27b9f56 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '85716f53e88048af9a8fdcbe4415979c'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:15:25.278578 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-ddd7071c-bd44-47f2-a699-6c9ae27b9f56 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '85716f53e88048af9a8fdcbe4415979c'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:15:25.298927 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1378/2773] 10.222.0.40 () {58 vars in 1224 bytes} [Fri Aug 26 11:15:25 2022] DELETE /identity/v3/users/85716f53e88048af9a8fdcbe4415979c => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 26 11:15:25.498988 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca360e6b-3901-4085-bf7c-803538bf76af service placement] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:15:25.522519 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca360e6b-3901-4085-bf7c-803538bf76af service placement] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:15:25.611568 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645fdee9-f3fa-4e33-b26c-b3fee8ac4237 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:15:25.614350 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645fdee9-f3fa-4e33-b26c-b3fee8ac4237 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:15:25.614705 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645fdee9-f3fa-4e33-b26c-b3fee8ac4237 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:15:25.615052 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645fdee9-f3fa-4e33-b26c-b3fee8ac4237 admin admin] PATH_INFO: `/v3/projects/f780402b93f94d26b2fceab83c0bf485` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:15:25.617382 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-645fdee9-f3fa-4e33-b26c-b3fee8ac4237 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f780402b93f94d26b2fceab83c0bf485)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:15:25.620157 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-645fdee9-f3fa-4e33-b26c-b3fee8ac4237 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:15:25.675673 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-645fdee9-f3fa-4e33-b26c-b3fee8ac4237 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f780402b93f94d26b2fceab83c0bf485', 'request_id': 'req-645fdee9-f3fa-4e33-b26c-b3fee8ac4237'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:15:25.683910 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-645fdee9-f3fa-4e33-b26c-b3fee8ac4237 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f780402b93f94d26b2fceab83c0bf485', 'request_id': 'req-645fdee9-f3fa-4e33-b26c-b3fee8ac4237'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:15:25.703942 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-645fdee9-f3fa-4e33-b26c-b3fee8ac4237 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f780402b93f94d26b2fceab83c0bf485', 'request_id': 'req-645fdee9-f3fa-4e33-b26c-b3fee8ac4237'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:15:25.724001 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-645fdee9-f3fa-4e33-b26c-b3fee8ac4237 admin admin] The token cache is being invalidate because project f780402b93f94d26b2fceab83c0bf485 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81424) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 11:15:25.724493 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-645fdee9-f3fa-4e33-b26c-b3fee8ac4237 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:15:25.725446 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-645fdee9-f3fa-4e33-b26c-b3fee8ac4237 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:15:25.728173 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1396/2774] 10.222.0.40 () {58 vars in 1233 bytes} [Fri Aug 26 11:15:25 2022] DELETE /identity/v3/projects/f780402b93f94d26b2fceab83c0bf485 => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 11:15:25.735563 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec17a38-ceb8-4ced-b6da-670bfaf00afd None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:15:25.736552 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec17a38-ceb8-4ced-b6da-670bfaf00afd None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:15:25.737056 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec17a38-ceb8-4ced-b6da-670bfaf00afd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:15:25.761372 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-1ec17a38-ceb8-4ced-b6da-670bfaf00afd None None] MFA Rules not processed for user `ca155a48e2024c32af3fd2e19e5fac11`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 11:15:25.762973 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-1ec17a38-ceb8-4ced-b6da-670bfaf00afd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:15:25.835406 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1379/2775] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 11:15:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 11:15:25.843640 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838b18fa-0254-4635-99a4-6b3bf05842ce None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:15:25.843994 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838b18fa-0254-4635-99a4-6b3bf05842ce None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:15:25.844348 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838b18fa-0254-4635-99a4-6b3bf05842ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:15:25.870773 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-838b18fa-0254-4635-99a4-6b3bf05842ce None None] MFA Rules not processed for user `ca155a48e2024c32af3fd2e19e5fac11`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 11:15:25.872110 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-838b18fa-0254-4635-99a4-6b3bf05842ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:15:25.882289 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1397/2776] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 11:15:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 11:15:25.886546 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec17a38-ceb8-4ced-b6da-670bfaf00afd None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:15:25.897787 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec17a38-ceb8-4ced-b6da-670bfaf00afd None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:15:25.904373 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4636dfa-426a-4aec-ac42-9fe2f5a9514c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:15:25.905128 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4636dfa-426a-4aec-ac42-9fe2f5a9514c admin admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:15:25.905506 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4636dfa-426a-4aec-ac42-9fe2f5a9514c admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:15:25.906106 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4636dfa-426a-4aec-ac42-9fe2f5a9514c admin admin] PATH_INFO: `/v3/domains` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:15:25.906905 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4636dfa-426a-4aec-ac42-9fe2f5a9514c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 11:15:25.907456 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4636dfa-426a-4aec-ac42-9fe2f5a9514c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:15:25.909955 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4636dfa-426a-4aec-ac42-9fe2f5a9514c admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:15:25.930188 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1380/2777] 10.222.0.40 () {58 vars in 1153 bytes} [Fri Aug 26 11:15:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 11:15:25.933018 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838b18fa-0254-4635-99a4-6b3bf05842ce None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:15:25.944978 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838b18fa-0254-4635-99a4-6b3bf05842ce None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:15:25.953555 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4c0d3b-c890-49c7-9f48-1e49f5a3a623 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:15:25.954517 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4c0d3b-c890-49c7-9f48-1e49f5a3a623 admin admin] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:15:25.954901 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4c0d3b-c890-49c7-9f48-1e49f5a3a623 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:15:25.955291 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4c0d3b-c890-49c7-9f48-1e49f5a3a623 admin admin] PATH_INFO: `/v3/projects` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:15:25.956070 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed4c0d3b-c890-49c7-9f48-1e49f5a3a623 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:15:25.958987 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed4c0d3b-c890-49c7-9f48-1e49f5a3a623 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:15:26.003257 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1398/2778] 10.222.0.40 () {60 vars in 1153 bytes} [Fri Aug 26 11:15:25 2022] POST /identity/v3/projects => generated 363 bytes in 70 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 26 11:15:26.007700 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4636dfa-426a-4aec-ac42-9fe2f5a9514c admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:15:26.018727 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4636dfa-426a-4aec-ac42-9fe2f5a9514c admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:15:26.080411 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 11:15:26.106364 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53eaff0b-fea8-47c9-a070-a3cd3976df88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:15:26.106979 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53eaff0b-fea8-47c9-a070-a3cd3976df88 admin admin] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:15:26.107257 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53eaff0b-fea8-47c9-a070-a3cd3976df88 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:15:26.107525 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53eaff0b-fea8-47c9-a070-a3cd3976df88 admin admin] PATH_INFO: `/v3/users` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:15:26.108193 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53eaff0b-fea8-47c9-a070-a3cd3976df88 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:15:26.110291 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53eaff0b-fea8-47c9-a070-a3cd3976df88 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:15:26.135439 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1381/2779] 10.222.0.40 () {60 vars in 1144 bytes} [Fri Aug 26 11:15:26 2022] POST /identity/v3/users => generated 282 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 11:15:26.139892 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4c0d3b-c890-49c7-9f48-1e49f5a3a623 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:15:26.151720 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4c0d3b-c890-49c7-9f48-1e49f5a3a623 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:15:26.156537 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4efc74-8b9e-440c-9d0d-4a446f377077 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:15:26.157217 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4efc74-8b9e-440c-9d0d-4a446f377077 admin admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:15:26.157564 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4efc74-8b9e-440c-9d0d-4a446f377077 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:15:26.157901 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4efc74-8b9e-440c-9d0d-4a446f377077 admin admin] PATH_INFO: `/v3/roles` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:15:26.158648 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f4efc74-8b9e-440c-9d0d-4a446f377077 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:15:26.161320 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f4efc74-8b9e-440c-9d0d-4a446f377077 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:15:26.177638 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1399/2780] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 26 11:15:26 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 11:15:26.181805 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53eaff0b-fea8-47c9-a070-a3cd3976df88 admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:15:26.194424 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53eaff0b-fea8-47c9-a070-a3cd3976df88 admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:15:26.199597 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f5e5fb-b4bb-4c09-aaeb-09a457e331c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:15:26.200587 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f5e5fb-b4bb-4c09-aaeb-09a457e331c4 admin admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:15:26.201267 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f5e5fb-b4bb-4c09-aaeb-09a457e331c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:15:26.201708 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f5e5fb-b4bb-4c09-aaeb-09a457e331c4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:15:26.202760 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f5e5fb-b4bb-4c09-aaeb-09a457e331c4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:15:26.204375 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f5e5fb-b4bb-4c09-aaeb-09a457e331c4 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:15:26.216279 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1382/2781] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 26 11:15:26 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 11:15:26.220484 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4efc74-8b9e-440c-9d0d-4a446f377077 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:15:26.233716 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4efc74-8b9e-440c-9d0d-4a446f377077 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:15:26.239797 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3f4913-207a-4185-b064-77ca775a41b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:15:26.240467 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3f4913-207a-4185-b064-77ca775a41b3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:15:26.240735 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3f4913-207a-4185-b064-77ca775a41b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:15:26.241001 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3f4913-207a-4185-b064-77ca775a41b3 admin admin] PATH_INFO: `/v3/projects/1a1f11aa68904fb2b21e60068f85128c/users/5b682d8c02b94c0b8cedd050b3520347/roles/23e174ce743b46eabe618c212691da55` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:15:26.259130 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af3f4913-207a-4185-b064-77ca775a41b3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1a1f11aa68904fb2b21e60068f85128c, user_id=5b682d8c02b94c0b8cedd050b3520347, role_id=23e174ce743b46eabe618c212691da55)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:15:26.261824 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af3f4913-207a-4185-b064-77ca775a41b3 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:15:26.282083 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1400/2782] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 26 11:15:26 2022] PUT /identity/v3/projects/1a1f11aa68904fb2b21e60068f85128c/users/5b682d8c02b94c0b8cedd050b3520347/roles/23e174ce743b46eabe618c212691da55 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 11:15:26.293183 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09bd830e-81d2-4385-a5e2-f2752ac66573 None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:15:26.293515 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09bd830e-81d2-4385-a5e2-f2752ac66573 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:15:26.293827 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09bd830e-81d2-4385-a5e2-f2752ac66573 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:15:26.326509 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-09bd830e-81d2-4385-a5e2-f2752ac66573 None None] MFA Rules not processed for user `5b682d8c02b94c0b8cedd050b3520347`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 11:15:26.327741 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-09bd830e-81d2-4385-a5e2-f2752ac66573 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:15:26.408199 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1383/2783] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 11:15:26 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 11:15:26.413350 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3f4913-207a-4185-b064-77ca775a41b3 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:15:26.424362 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 11:15:26.490096 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3f4913-207a-4185-b064-77ca775a41b3 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:15:26.495730 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb8651f-d29a-49ef-9e34-e90939eb6aa9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:15:26.496346 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb8651f-d29a-49ef-9e34-e90939eb6aa9 admin admin] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:15:26.496619 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb8651f-d29a-49ef-9e34-e90939eb6aa9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:15:26.496884 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb8651f-d29a-49ef-9e34-e90939eb6aa9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:15:26.497509 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fb8651f-d29a-49ef-9e34-e90939eb6aa9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:15:26.499432 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fb8651f-d29a-49ef-9e34-e90939eb6aa9 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:15:26.531342 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1401/2784] 10.222.0.40 () {60 vars in 1153 bytes} [Fri Aug 26 11:15:26 2022] POST /identity/v3/projects => generated 365 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 11:15:26.532435 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09bd830e-81d2-4385-a5e2-f2752ac66573 None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:15:26.544486 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09bd830e-81d2-4385-a5e2-f2752ac66573 None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:15:26.604408 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c290120f-98e2-42ef-bcb9-a902018b5c49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:15:26.605127 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c290120f-98e2-42ef-bcb9-a902018b5c49 admin admin] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:15:26.605309 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c290120f-98e2-42ef-bcb9-a902018b5c49 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:15:26.605595 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c290120f-98e2-42ef-bcb9-a902018b5c49 admin admin] PATH_INFO: `/v3/users` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:15:26.606304 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c290120f-98e2-42ef-bcb9-a902018b5c49 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:15:26.608205 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c290120f-98e2-42ef-bcb9-a902018b5c49 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:15:26.633801 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1384/2785] 10.222.0.40 () {60 vars in 1144 bytes} [Fri Aug 26 11:15:26 2022] POST /identity/v3/users => generated 289 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 26 11:15:26.638133 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb8651f-d29a-49ef-9e34-e90939eb6aa9 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:15:26.649380 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb8651f-d29a-49ef-9e34-e90939eb6aa9 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:15:26.654058 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0435d624-670f-43e5-bd28-9e638a49d6bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:15:26.654671 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0435d624-670f-43e5-bd28-9e638a49d6bf admin admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:15:26.654950 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0435d624-670f-43e5-bd28-9e638a49d6bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:15:26.655218 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0435d624-670f-43e5-bd28-9e638a49d6bf admin admin] PATH_INFO: `/v3/roles` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:15:26.655865 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0435d624-670f-43e5-bd28-9e638a49d6bf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:15:26.657473 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0435d624-670f-43e5-bd28-9e638a49d6bf admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:15:26.674410 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1402/2786] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 26 11:15:26 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 11:15:26.675877 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c290120f-98e2-42ef-bcb9-a902018b5c49 admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:15:26.686999 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c290120f-98e2-42ef-bcb9-a902018b5c49 admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:15:26.692739 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f47c6a2-c681-41cd-a6f0-172c12f37a8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:15:26.693601 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f47c6a2-c681-41cd-a6f0-172c12f37a8c admin admin] REQUEST_METHOD: `PUT` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:15:26.693882 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f47c6a2-c681-41cd-a6f0-172c12f37a8c admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:15:26.694194 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f47c6a2-c681-41cd-a6f0-172c12f37a8c admin admin] PATH_INFO: `/v3/projects/6560d0cd8f8f4606b5b0c7a453ca0295/users/72b361ef25f14240bad0abe3a18ad9f6/roles/1f97e4602e2e4d6498913a81f9ae8d6a` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:15:26.714903 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f47c6a2-c681-41cd-a6f0-172c12f37a8c admin admin] RBAC: Authorizing `identity:create_grant(project_id=6560d0cd8f8f4606b5b0c7a453ca0295, user_id=72b361ef25f14240bad0abe3a18ad9f6, role_id=1f97e4602e2e4d6498913a81f9ae8d6a)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:15:26.719608 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f47c6a2-c681-41cd-a6f0-172c12f37a8c admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:15:26.740133 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1385/2787] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 26 11:15:26 2022] PUT /identity/v3/projects/6560d0cd8f8f4606b5b0c7a453ca0295/users/72b361ef25f14240bad0abe3a18ad9f6/roles/1f97e4602e2e4d6498913a81f9ae8d6a => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 11:15:26.746679 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0435d624-670f-43e5-bd28-9e638a49d6bf admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:15:26.758322 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0435d624-670f-43e5-bd28-9e638a49d6bf admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:15:26.824140 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3200d7-4df8-4ffe-8a17-1827b43239ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:15:26.824755 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3200d7-4df8-4ffe-8a17-1827b43239ce admin admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:15:26.825117 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3200d7-4df8-4ffe-8a17-1827b43239ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:15:26.825374 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3200d7-4df8-4ffe-8a17-1827b43239ce admin admin] PATH_INFO: `/v3/roles` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:15:26.826068 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be3200d7-4df8-4ffe-8a17-1827b43239ce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:15:26.827662 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be3200d7-4df8-4ffe-8a17-1827b43239ce admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:15:26.845168 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1403/2788] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 26 11:15:26 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 26 11:15:26.847127 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f47c6a2-c681-41cd-a6f0-172c12f37a8c admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:15:26.861839 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f47c6a2-c681-41cd-a6f0-172c12f37a8c admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:15:26.867907 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6634a9b-03a0-4eb2-898c-0373a7369c12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:15:26.868792 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6634a9b-03a0-4eb2-898c-0373a7369c12 admin admin] REQUEST_METHOD: `PUT` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:15:26.869065 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6634a9b-03a0-4eb2-898c-0373a7369c12 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:15:26.869336 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6634a9b-03a0-4eb2-898c-0373a7369c12 admin admin] PATH_INFO: `/v3/projects/6560d0cd8f8f4606b5b0c7a453ca0295/users/72b361ef25f14240bad0abe3a18ad9f6/roles/1f97e4602e2e4d6498913a81f9ae8d6a` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:15:26.879595 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6634a9b-03a0-4eb2-898c-0373a7369c12 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6560d0cd8f8f4606b5b0c7a453ca0295, user_id=72b361ef25f14240bad0abe3a18ad9f6, role_id=1f97e4602e2e4d6498913a81f9ae8d6a)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:15:26.882342 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6634a9b-03a0-4eb2-898c-0373a7369c12 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:15:26.895708 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1386/2789] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 26 11:15:26 2022] PUT /identity/v3/projects/6560d0cd8f8f4606b5b0c7a453ca0295/users/72b361ef25f14240bad0abe3a18ad9f6/roles/1f97e4602e2e4d6498913a81f9ae8d6a => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 11:15:26.906878 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6187514-ad0b-4d4a-a452-0a622b547817 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:15:26.907253 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6187514-ad0b-4d4a-a452-0a622b547817 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:15:26.907465 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6187514-ad0b-4d4a-a452-0a622b547817 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:15:26.943534 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-a6187514-ad0b-4d4a-a452-0a622b547817 None None] MFA Rules not processed for user `72b361ef25f14240bad0abe3a18ad9f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 11:15:26.944787 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-a6187514-ad0b-4d4a-a452-0a622b547817 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:15:27.066077 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1404/2790] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 11:15:26 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 11:15:27.079333 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6634a9b-03a0-4eb2-898c-0373a7369c12 admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:15:27.082719 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-c6634a9b-03a0-4eb2-898c-0373a7369c12 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:15:27.093203 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 11:15:27.178511 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6634a9b-03a0-4eb2-898c-0373a7369c12 admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:15:27.183055 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1479c2-c698-439c-b36a-39c990f1b341 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c277338a7f814cc6b01c5419775031bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e30ccf0c1204727b786876c141dbcd2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:15:27.183850 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1479c2-c698-439c-b36a-39c990f1b341 service cinder] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:15:27.184068 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1479c2-c698-439c-b36a-39c990f1b341 service cinder] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:15:27.184370 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1479c2-c698-439c-b36a-39c990f1b341 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:15:27.197523 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c1479c2-c698-439c-b36a-39c990f1b341 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:15:27.199003 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c1479c2-c698-439c-b36a-39c990f1b341 service cinder] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:15:27.210063 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1387/2791] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 11:15:27 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 11:15:27.730117 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6187514-ad0b-4d4a-a452-0a622b547817 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:15:27.738404 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 11:15:27.814518 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6187514-ad0b-4d4a-a452-0a622b547817 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:15:27.819933 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114c7463-3bd3-4042-8743-7f34a866f624 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:15:27.820570 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114c7463-3bd3-4042-8743-7f34a866f624 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:15:27.820847 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114c7463-3bd3-4042-8743-7f34a866f624 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:15:27.821120 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114c7463-3bd3-4042-8743-7f34a866f624 admin admin] PATH_INFO: `/v3/users/5b682d8c02b94c0b8cedd050b3520347` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:15:27.827835 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-114c7463-3bd3-4042-8743-7f34a866f624 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5b682d8c02b94c0b8cedd050b3520347)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:15:27.829911 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-114c7463-3bd3-4042-8743-7f34a866f624 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:15:27.896357 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-114c7463-3bd3-4042-8743-7f34a866f624 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5b682d8c02b94c0b8cedd050b3520347'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:15:27.897126 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-114c7463-3bd3-4042-8743-7f34a866f624 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5b682d8c02b94c0b8cedd050b3520347'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:15:27.907314 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-114c7463-3bd3-4042-8743-7f34a866f624 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5b682d8c02b94c0b8cedd050b3520347'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:15:27.908382 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-114c7463-3bd3-4042-8743-7f34a866f624 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5b682d8c02b94c0b8cedd050b3520347'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:15:27.937382 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-114c7463-3bd3-4042-8743-7f34a866f624 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5b682d8c02b94c0b8cedd050b3520347'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:15:27.956794 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1405/2792] 10.222.0.40 () {58 vars in 1224 bytes} [Fri Aug 26 11:15:27 2022] DELETE /identity/v3/users/5b682d8c02b94c0b8cedd050b3520347 => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 11:15:27.962218 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6598a0-7316-4721-bade-240e3860814b None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:15:27.962711 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6598a0-7316-4721-bade-240e3860814b None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:15:27.962903 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6598a0-7316-4721-bade-240e3860814b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:15:27.986281 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-ba6598a0-7316-4721-bade-240e3860814b None None] MFA Rules not processed for user `ca155a48e2024c32af3fd2e19e5fac11`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 11:15:27.988418 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-ba6598a0-7316-4721-bade-240e3860814b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:15:28.059255 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1388/2793] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 11:15:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 11:15:28.069658 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114c7463-3bd3-4042-8743-7f34a866f624 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:15:28.071344 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-114c7463-3bd3-4042-8743-7f34a866f624 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:15:28.140142 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114c7463-3bd3-4042-8743-7f34a866f624 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:15:28.145529 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4709cb72-b2c5-4852-a464-1a1b7b19744d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91da2acd2534423091537e1ba068d2b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e30ccf0c1204727b786876c141dbcd2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:15:28.146344 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4709cb72-b2c5-4852-a464-1a1b7b19744d service neutron] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:15:28.146628 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4709cb72-b2c5-4852-a464-1a1b7b19744d service neutron] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:15:28.146903 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4709cb72-b2c5-4852-a464-1a1b7b19744d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:15:28.157288 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4709cb72-b2c5-4852-a464-1a1b7b19744d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:15:28.160579 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4709cb72-b2c5-4852-a464-1a1b7b19744d service neutron] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:15:28.170240 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1406/2794] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 11:15:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 11:15:28.726097 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6598a0-7316-4721-bade-240e3860814b None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:15:28.727537 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-ba6598a0-7316-4721-bade-240e3860814b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:15:28.739830 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6598a0-7316-4721-bade-240e3860814b None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:15:28.745767 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e960e08f-ff2a-40df-a265-500c35a4d7e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:15:28.746704 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e960e08f-ff2a-40df-a265-500c35a4d7e3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:15:28.746975 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e960e08f-ff2a-40df-a265-500c35a4d7e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:15:28.747242 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e960e08f-ff2a-40df-a265-500c35a4d7e3 admin admin] PATH_INFO: `/v3/projects/1a1f11aa68904fb2b21e60068f85128c` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:15:28.748404 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e960e08f-ff2a-40df-a265-500c35a4d7e3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1a1f11aa68904fb2b21e60068f85128c)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:15:28.750612 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e960e08f-ff2a-40df-a265-500c35a4d7e3 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:15:28.811913 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-e960e08f-ff2a-40df-a265-500c35a4d7e3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1a1f11aa68904fb2b21e60068f85128c', 'request_id': 'req-e960e08f-ff2a-40df-a265-500c35a4d7e3'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:15:28.819373 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-e960e08f-ff2a-40df-a265-500c35a4d7e3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1a1f11aa68904fb2b21e60068f85128c', 'request_id': 'req-e960e08f-ff2a-40df-a265-500c35a4d7e3'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:15:28.828343 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-e960e08f-ff2a-40df-a265-500c35a4d7e3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1a1f11aa68904fb2b21e60068f85128c', 'request_id': 'req-e960e08f-ff2a-40df-a265-500c35a4d7e3'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:15:28.842633 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-e960e08f-ff2a-40df-a265-500c35a4d7e3 admin admin] The token cache is being invalidate because project 1a1f11aa68904fb2b21e60068f85128c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81423) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 11:15:28.842945 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-e960e08f-ff2a-40df-a265-500c35a4d7e3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:15:28.843667 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-e960e08f-ff2a-40df-a265-500c35a4d7e3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:15:28.845231 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1389/2795] 10.222.0.40 () {58 vars in 1233 bytes} [Fri Aug 26 11:15:28 2022] DELETE /identity/v3/projects/1a1f11aa68904fb2b21e60068f85128c => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 11:15:28.850200 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4709cb72-b2c5-4852-a464-1a1b7b19744d service neutron] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:15:28.860534 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4709cb72-b2c5-4852-a464-1a1b7b19744d service neutron] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:15:28.890425 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 11:15:28.938178 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c2e981-9213-4e84-9d64-d5b0948c4517 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:15:28.938832 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c2e981-9213-4e84-9d64-d5b0948c4517 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:15:28.939096 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c2e981-9213-4e84-9d64-d5b0948c4517 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:15:28.939363 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c2e981-9213-4e84-9d64-d5b0948c4517 admin admin] PATH_INFO: `/v3/users/72b361ef25f14240bad0abe3a18ad9f6` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:15:28.941398 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c2e981-9213-4e84-9d64-d5b0948c4517 admin admin] RBAC: Authorizing `identity:delete_user(user_id=72b361ef25f14240bad0abe3a18ad9f6)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:15:28.943363 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c2e981-9213-4e84-9d64-d5b0948c4517 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:15:29.011464 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-96c2e981-9213-4e84-9d64-d5b0948c4517 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '72b361ef25f14240bad0abe3a18ad9f6'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:15:29.012156 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-96c2e981-9213-4e84-9d64-d5b0948c4517 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '72b361ef25f14240bad0abe3a18ad9f6'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:15:29.023343 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-96c2e981-9213-4e84-9d64-d5b0948c4517 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '72b361ef25f14240bad0abe3a18ad9f6'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:15:29.024250 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-96c2e981-9213-4e84-9d64-d5b0948c4517 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '72b361ef25f14240bad0abe3a18ad9f6'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:15:29.040823 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-96c2e981-9213-4e84-9d64-d5b0948c4517 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '72b361ef25f14240bad0abe3a18ad9f6'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:15:29.058661 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1407/2796] 10.222.0.40 () {58 vars in 1224 bytes} [Fri Aug 26 11:15:28 2022] DELETE /identity/v3/users/72b361ef25f14240bad0abe3a18ad9f6 => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 11:15:29.383096 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e960e08f-ff2a-40df-a265-500c35a4d7e3 admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:15:29.385328 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-e960e08f-ff2a-40df-a265-500c35a4d7e3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:15:29.403443 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e960e08f-ff2a-40df-a265-500c35a4d7e3 admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:15:29.435766 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 11:15:29.474576 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d822a8-b178-48de-bc26-fb2c907bcc52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:15:29.475214 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d822a8-b178-48de-bc26-fb2c907bcc52 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:15:29.475493 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d822a8-b178-48de-bc26-fb2c907bcc52 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:15:29.475761 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d822a8-b178-48de-bc26-fb2c907bcc52 admin admin] PATH_INFO: `/v3/projects/6560d0cd8f8f4606b5b0c7a453ca0295` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:15:29.479448 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d822a8-b178-48de-bc26-fb2c907bcc52 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6560d0cd8f8f4606b5b0c7a453ca0295)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:15:29.480425 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d822a8-b178-48de-bc26-fb2c907bcc52 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:15:29.537844 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-44d822a8-b178-48de-bc26-fb2c907bcc52 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6560d0cd8f8f4606b5b0c7a453ca0295', 'request_id': 'req-44d822a8-b178-48de-bc26-fb2c907bcc52'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:15:29.545962 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-44d822a8-b178-48de-bc26-fb2c907bcc52 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6560d0cd8f8f4606b5b0c7a453ca0295', 'request_id': 'req-44d822a8-b178-48de-bc26-fb2c907bcc52'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:15:29.554078 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-44d822a8-b178-48de-bc26-fb2c907bcc52 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6560d0cd8f8f4606b5b0c7a453ca0295', 'request_id': 'req-44d822a8-b178-48de-bc26-fb2c907bcc52'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:15:29.570656 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-44d822a8-b178-48de-bc26-fb2c907bcc52 admin admin] The token cache is being invalidate because project 6560d0cd8f8f4606b5b0c7a453ca0295 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81423) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 11:15:29.571262 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-44d822a8-b178-48de-bc26-fb2c907bcc52 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:15:29.572072 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-44d822a8-b178-48de-bc26-fb2c907bcc52 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:15:29.577589 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1390/2797] 10.222.0.40 () {58 vars in 1233 bytes} [Fri Aug 26 11:15:29 2022] DELETE /identity/v3/projects/6560d0cd8f8f4606b5b0c7a453ca0295 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 11:15:29.580170 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cec973-ce9c-4aba-9cdd-a7bb9e3b6d21 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:15:29.580563 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cec973-ce9c-4aba-9cdd-a7bb9e3b6d21 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:15:29.581599 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cec973-ce9c-4aba-9cdd-a7bb9e3b6d21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:15:29.606047 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-f5cec973-ce9c-4aba-9cdd-a7bb9e3b6d21 None None] MFA Rules not processed for user `ca155a48e2024c32af3fd2e19e5fac11`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 11:15:29.607459 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-f5cec973-ce9c-4aba-9cdd-a7bb9e3b6d21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:15:29.691956 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1408/2798] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 11:15:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 11:15:29.699950 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36d8b78-a2d1-48ae-bcf3-ba85b06545e1 None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:15:29.700247 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36d8b78-a2d1-48ae-bcf3-ba85b06545e1 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:15:29.700789 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36d8b78-a2d1-48ae-bcf3-ba85b06545e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:15:29.723800 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-c36d8b78-a2d1-48ae-bcf3-ba85b06545e1 None None] MFA Rules not processed for user `ca155a48e2024c32af3fd2e19e5fac11`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 11:15:29.725318 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-c36d8b78-a2d1-48ae-bcf3-ba85b06545e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:15:29.733672 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1391/2799] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 11:15:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 11:15:29.738272 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5cec973-ce9c-4aba-9cdd-a7bb9e3b6d21 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:15:29.748345 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5cec973-ce9c-4aba-9cdd-a7bb9e3b6d21 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:15:29.753870 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf8606d-65c1-4821-9498-e77ab92fe3d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:15:29.754516 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf8606d-65c1-4821-9498-e77ab92fe3d2 admin admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:15:29.754791 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf8606d-65c1-4821-9498-e77ab92fe3d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:15:29.755056 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf8606d-65c1-4821-9498-e77ab92fe3d2 admin admin] PATH_INFO: `/v3/domains` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:15:29.755708 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf8606d-65c1-4821-9498-e77ab92fe3d2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 11:15:29.756005 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf8606d-65c1-4821-9498-e77ab92fe3d2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:15:29.757604 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf8606d-65c1-4821-9498-e77ab92fe3d2 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:15:29.770521 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1409/2800] 10.222.0.40 () {58 vars in 1153 bytes} [Fri Aug 26 11:15:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 11:15:29.776298 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36d8b78-a2d1-48ae-bcf3-ba85b06545e1 None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:15:29.787921 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36d8b78-a2d1-48ae-bcf3-ba85b06545e1 None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:15:29.792817 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095d39e9-f78c-46aa-b3b0-7fe0e5c5ffbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:15:29.793437 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095d39e9-f78c-46aa-b3b0-7fe0e5c5ffbd admin admin] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:15:29.793788 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095d39e9-f78c-46aa-b3b0-7fe0e5c5ffbd admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:15:29.794102 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095d39e9-f78c-46aa-b3b0-7fe0e5c5ffbd admin admin] PATH_INFO: `/v3/projects` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:15:29.794748 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-095d39e9-f78c-46aa-b3b0-7fe0e5c5ffbd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:15:29.796672 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-095d39e9-f78c-46aa-b3b0-7fe0e5c5ffbd admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:15:29.832684 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1392/2801] 10.222.0.40 () {60 vars in 1153 bytes} [Fri Aug 26 11:15:29 2022] POST /identity/v3/projects => generated 369 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 11:15:29.833837 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf8606d-65c1-4821-9498-e77ab92fe3d2 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:15:29.844415 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf8606d-65c1-4821-9498-e77ab92fe3d2 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:15:29.878976 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 11:15:29.912273 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7dbcfc-06b3-4dd6-8201-6cac54a4aaec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:15:29.912890 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7dbcfc-06b3-4dd6-8201-6cac54a4aaec admin admin] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:15:29.913173 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7dbcfc-06b3-4dd6-8201-6cac54a4aaec admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:15:29.913445 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7dbcfc-06b3-4dd6-8201-6cac54a4aaec admin admin] PATH_INFO: `/v3/users` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:15:29.914106 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb7dbcfc-06b3-4dd6-8201-6cac54a4aaec admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:15:29.916034 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb7dbcfc-06b3-4dd6-8201-6cac54a4aaec admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:15:29.938334 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1410/2802] 10.222.0.40 () {60 vars in 1144 bytes} [Fri Aug 26 11:15:29 2022] POST /identity/v3/users => generated 285 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 11:15:29.945047 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095d39e9-f78c-46aa-b3b0-7fe0e5c5ffbd admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:15:29.956352 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095d39e9-f78c-46aa-b3b0-7fe0e5c5ffbd admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:15:29.961094 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce29166-8535-4571-8d1e-72b8220101ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:15:29.961693 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce29166-8535-4571-8d1e-72b8220101ce admin admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:15:29.961970 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce29166-8535-4571-8d1e-72b8220101ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:15:29.962270 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce29166-8535-4571-8d1e-72b8220101ce admin admin] PATH_INFO: `/v3/roles` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:15:29.962913 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce29166-8535-4571-8d1e-72b8220101ce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:15:29.964486 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce29166-8535-4571-8d1e-72b8220101ce admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:15:29.975308 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1393/2803] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 26 11:15:29 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 26 11:15:29.979845 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7dbcfc-06b3-4dd6-8201-6cac54a4aaec admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:15:29.991372 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7dbcfc-06b3-4dd6-8201-6cac54a4aaec admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:15:29.996444 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de14452e-8abe-46fd-8c05-6a22fc9a9880 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:15:29.997049 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de14452e-8abe-46fd-8c05-6a22fc9a9880 admin admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:15:29.997323 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de14452e-8abe-46fd-8c05-6a22fc9a9880 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:15:29.997590 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de14452e-8abe-46fd-8c05-6a22fc9a9880 admin admin] PATH_INFO: `/v3/roles` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:15:29.998263 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de14452e-8abe-46fd-8c05-6a22fc9a9880 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:15:29.999845 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de14452e-8abe-46fd-8c05-6a22fc9a9880 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:15:30.013160 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1411/2804] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 26 11:15:29 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 11:15:30.017566 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce29166-8535-4571-8d1e-72b8220101ce admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:15:30.028783 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce29166-8535-4571-8d1e-72b8220101ce admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:15:30.033650 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc7aef6-9ae7-49f0-aff3-96a12aa3ac8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:15:30.034882 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc7aef6-9ae7-49f0-aff3-96a12aa3ac8d admin admin] REQUEST_METHOD: `PUT` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:15:30.035154 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc7aef6-9ae7-49f0-aff3-96a12aa3ac8d admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:15:30.035422 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc7aef6-9ae7-49f0-aff3-96a12aa3ac8d admin admin] PATH_INFO: `/v3/projects/3e93d0cf8cb54b04a4f51b392e6c7783/users/ddfa433107d24ea68f8ff4c38f2ed9a9/roles/23e174ce743b46eabe618c212691da55` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:15:30.051330 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc7aef6-9ae7-49f0-aff3-96a12aa3ac8d admin admin] RBAC: Authorizing `identity:create_grant(project_id=3e93d0cf8cb54b04a4f51b392e6c7783, user_id=ddfa433107d24ea68f8ff4c38f2ed9a9, role_id=23e174ce743b46eabe618c212691da55)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:15:30.053995 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc7aef6-9ae7-49f0-aff3-96a12aa3ac8d admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:15:30.067565 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1394/2805] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 26 11:15:30 2022] PUT /identity/v3/projects/3e93d0cf8cb54b04a4f51b392e6c7783/users/ddfa433107d24ea68f8ff4c38f2ed9a9/roles/23e174ce743b46eabe618c212691da55 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 11:15:30.079055 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ce1c0f-6fbb-4818-a86a-d6c10e5b12ff None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:15:30.079334 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ce1c0f-6fbb-4818-a86a-d6c10e5b12ff None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:15:30.079600 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ce1c0f-6fbb-4818-a86a-d6c10e5b12ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:15:30.111402 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-c6ce1c0f-6fbb-4818-a86a-d6c10e5b12ff None None] MFA Rules not processed for user `ddfa433107d24ea68f8ff4c38f2ed9a9`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 11:15:30.113282 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-c6ce1c0f-6fbb-4818-a86a-d6c10e5b12ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:15:30.185728 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1412/2806] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 11:15:30 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 11:15:30.190409 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc7aef6-9ae7-49f0-aff3-96a12aa3ac8d admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:15:30.252546 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 11:15:30.260158 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc7aef6-9ae7-49f0-aff3-96a12aa3ac8d admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:15:30.272316 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e00471-6a87-4a27-8ccb-57d4364f0c78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:15:30.272951 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e00471-6a87-4a27-8ccb-57d4364f0c78 admin admin] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:15:30.273227 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e00471-6a87-4a27-8ccb-57d4364f0c78 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:15:30.273496 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e00471-6a87-4a27-8ccb-57d4364f0c78 admin admin] PATH_INFO: `/v3/projects` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:15:30.274191 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e00471-6a87-4a27-8ccb-57d4364f0c78 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:15:30.276149 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e00471-6a87-4a27-8ccb-57d4364f0c78 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:15:30.307342 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1395/2807] 10.222.0.40 () {60 vars in 1153 bytes} [Fri Aug 26 11:15:30 2022] POST /identity/v3/projects => generated 367 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 11:15:30.311565 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ce1c0f-6fbb-4818-a86a-d6c10e5b12ff None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:15:30.322572 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ce1c0f-6fbb-4818-a86a-d6c10e5b12ff None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:15:30.384092 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501f2847-975a-4277-a527-f64933786eca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:15:30.384714 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501f2847-975a-4277-a527-f64933786eca admin admin] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:15:30.384997 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501f2847-975a-4277-a527-f64933786eca admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:15:30.385273 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501f2847-975a-4277-a527-f64933786eca admin admin] PATH_INFO: `/v3/users` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:15:30.385903 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-501f2847-975a-4277-a527-f64933786eca admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:15:30.387856 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-501f2847-975a-4277-a527-f64933786eca admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:15:30.411166 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1413/2808] 10.222.0.40 () {60 vars in 1144 bytes} [Fri Aug 26 11:15:30 2022] POST /identity/v3/users => generated 290 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 11:15:30.415438 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e00471-6a87-4a27-8ccb-57d4364f0c78 admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:15:30.426867 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e00471-6a87-4a27-8ccb-57d4364f0c78 admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:15:30.433681 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4669aad-d0ae-49d3-bb75-3996bad2fb9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:15:30.434538 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4669aad-d0ae-49d3-bb75-3996bad2fb9f admin admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:15:30.434909 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4669aad-d0ae-49d3-bb75-3996bad2fb9f admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:15:30.435276 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4669aad-d0ae-49d3-bb75-3996bad2fb9f admin admin] PATH_INFO: `/v3/roles` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:15:30.436045 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4669aad-d0ae-49d3-bb75-3996bad2fb9f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:15:30.438152 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4669aad-d0ae-49d3-bb75-3996bad2fb9f admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:15:30.450570 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1396/2809] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 26 11:15:30 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 26 11:15:30.454632 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501f2847-975a-4277-a527-f64933786eca admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:15:30.465684 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501f2847-975a-4277-a527-f64933786eca admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:15:30.470364 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2bfc090-ed94-4b99-9e0e-675c5c33e24c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:15:30.471008 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bfc090-ed94-4b99-9e0e-675c5c33e24c admin admin] REQUEST_METHOD: `PUT` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:15:30.471288 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bfc090-ed94-4b99-9e0e-675c5c33e24c admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:15:30.472317 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bfc090-ed94-4b99-9e0e-675c5c33e24c admin admin] PATH_INFO: `/v3/projects/91496cfd4dbe4c3890a3688f1f54d8e3/users/73c4b6869c7c4e35af0e489b044929da/roles/1f97e4602e2e4d6498913a81f9ae8d6a` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:15:30.489075 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2bfc090-ed94-4b99-9e0e-675c5c33e24c admin admin] RBAC: Authorizing `identity:create_grant(project_id=91496cfd4dbe4c3890a3688f1f54d8e3, user_id=73c4b6869c7c4e35af0e489b044929da, role_id=1f97e4602e2e4d6498913a81f9ae8d6a)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:15:30.491801 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2bfc090-ed94-4b99-9e0e-675c5c33e24c admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:15:30.508701 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1414/2810] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 26 11:15:30 2022] PUT /identity/v3/projects/91496cfd4dbe4c3890a3688f1f54d8e3/users/73c4b6869c7c4e35af0e489b044929da/roles/1f97e4602e2e4d6498913a81f9ae8d6a => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 11:15:30.513141 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4669aad-d0ae-49d3-bb75-3996bad2fb9f admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:15:30.526243 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4669aad-d0ae-49d3-bb75-3996bad2fb9f admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:15:30.595938 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bbf8974-6219-46ff-8126-0587e1346a97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:15:30.597962 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbf8974-6219-46ff-8126-0587e1346a97 admin admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:15:30.598276 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbf8974-6219-46ff-8126-0587e1346a97 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:15:30.598547 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbf8974-6219-46ff-8126-0587e1346a97 admin admin] PATH_INFO: `/v3/roles` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:15:30.599203 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bbf8974-6219-46ff-8126-0587e1346a97 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:15:30.600840 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bbf8974-6219-46ff-8126-0587e1346a97 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:15:30.611639 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1397/2811] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 26 11:15:30 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 26 11:15:30.615901 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2bfc090-ed94-4b99-9e0e-675c5c33e24c admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:15:30.626872 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2bfc090-ed94-4b99-9e0e-675c5c33e24c admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:15:30.632693 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c480ac-d883-4fe8-9f1a-095b735f76a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:15:30.633375 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c480ac-d883-4fe8-9f1a-095b735f76a7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:15:30.633653 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c480ac-d883-4fe8-9f1a-095b735f76a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:15:30.633931 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c480ac-d883-4fe8-9f1a-095b735f76a7 admin admin] PATH_INFO: `/v3/projects/91496cfd4dbe4c3890a3688f1f54d8e3/users/73c4b6869c7c4e35af0e489b044929da/roles/1f97e4602e2e4d6498913a81f9ae8d6a` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:15:30.636010 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c480ac-d883-4fe8-9f1a-095b735f76a7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=91496cfd4dbe4c3890a3688f1f54d8e3, user_id=73c4b6869c7c4e35af0e489b044929da, role_id=1f97e4602e2e4d6498913a81f9ae8d6a)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:15:30.638656 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c480ac-d883-4fe8-9f1a-095b735f76a7 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:15:30.641124 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 11:15:30.654377 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1415/2812] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 26 11:15:30 2022] PUT /identity/v3/projects/91496cfd4dbe4c3890a3688f1f54d8e3/users/73c4b6869c7c4e35af0e489b044929da/roles/1f97e4602e2e4d6498913a81f9ae8d6a => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 11:15:30.666045 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078ea04f-7c2b-4cd4-a03a-919798def7e0 None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:15:30.666399 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078ea04f-7c2b-4cd4-a03a-919798def7e0 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:15:30.666776 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078ea04f-7c2b-4cd4-a03a-919798def7e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:15:30.701469 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-078ea04f-7c2b-4cd4-a03a-919798def7e0 None None] MFA Rules not processed for user `73c4b6869c7c4e35af0e489b044929da`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 11:15:30.702745 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-078ea04f-7c2b-4cd4-a03a-919798def7e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:15:30.783185 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1398/2813] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 11:15:30 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 11:15:30.795483 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c480ac-d883-4fe8-9f1a-095b735f76a7 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:15:30.797254 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-43c480ac-d883-4fe8-9f1a-095b735f76a7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:15:30.864592 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c480ac-d883-4fe8-9f1a-095b735f76a7 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:15:30.868208 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcad3631-3336-44e8-b146-6b99139cabbd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c277338a7f814cc6b01c5419775031bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e30ccf0c1204727b786876c141dbcd2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:15:30.868811 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcad3631-3336-44e8-b146-6b99139cabbd service cinder] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:15:30.869086 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcad3631-3336-44e8-b146-6b99139cabbd service cinder] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:15:30.869355 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcad3631-3336-44e8-b146-6b99139cabbd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:15:30.879760 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcad3631-3336-44e8-b146-6b99139cabbd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:15:30.881224 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcad3631-3336-44e8-b146-6b99139cabbd service cinder] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:15:30.891902 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1416/2814] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 11:15:30 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 11:15:32.373432 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078ea04f-7c2b-4cd4-a03a-919798def7e0 None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:15:32.432259 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 11:15:32.447539 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078ea04f-7c2b-4cd4-a03a-919798def7e0 None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:15:32.452734 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d67fe10-ff57-4e8b-bc4b-82741ab58143 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:15:32.453391 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d67fe10-ff57-4e8b-bc4b-82741ab58143 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:15:32.453675 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d67fe10-ff57-4e8b-bc4b-82741ab58143 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:15:32.453966 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d67fe10-ff57-4e8b-bc4b-82741ab58143 admin admin] PATH_INFO: `/v3/users/ddfa433107d24ea68f8ff4c38f2ed9a9` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:15:32.455173 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d67fe10-ff57-4e8b-bc4b-82741ab58143 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ddfa433107d24ea68f8ff4c38f2ed9a9)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:15:32.457272 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d67fe10-ff57-4e8b-bc4b-82741ab58143 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:15:32.509738 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-2d67fe10-ff57-4e8b-bc4b-82741ab58143 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ddfa433107d24ea68f8ff4c38f2ed9a9'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:15:32.518661 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-2d67fe10-ff57-4e8b-bc4b-82741ab58143 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ddfa433107d24ea68f8ff4c38f2ed9a9'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:15:32.519598 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-2d67fe10-ff57-4e8b-bc4b-82741ab58143 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ddfa433107d24ea68f8ff4c38f2ed9a9'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:15:32.532304 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-2d67fe10-ff57-4e8b-bc4b-82741ab58143 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ddfa433107d24ea68f8ff4c38f2ed9a9'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:15:32.533039 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-2d67fe10-ff57-4e8b-bc4b-82741ab58143 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ddfa433107d24ea68f8ff4c38f2ed9a9'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:15:32.552771 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1399/2815] 10.222.0.40 () {58 vars in 1224 bytes} [Fri Aug 26 11:15:32 2022] DELETE /identity/v3/users/ddfa433107d24ea68f8ff4c38f2ed9a9 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 11:15:32.554749 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d5edfa-d20c-435f-9a20-678f952483de None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:15:32.555600 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d5edfa-d20c-435f-9a20-678f952483de None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:15:32.555945 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d5edfa-d20c-435f-9a20-678f952483de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:15:32.583226 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-63d5edfa-d20c-435f-9a20-678f952483de None None] MFA Rules not processed for user `ca155a48e2024c32af3fd2e19e5fac11`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 11:15:32.584498 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-63d5edfa-d20c-435f-9a20-678f952483de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:15:32.605209 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d67fe10-ff57-4e8b-bc4b-82741ab58143 admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:15:32.607023 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-2d67fe10-ff57-4e8b-bc4b-82741ab58143 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:15:32.662688 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1417/2816] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 11:15:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 11:15:32.680645 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d5edfa-d20c-435f-9a20-678f952483de None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:15:32.682646 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-63d5edfa-d20c-435f-9a20-678f952483de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:15:32.728615 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d67fe10-ff57-4e8b-bc4b-82741ab58143 admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:15:32.734527 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5afc883-3ba8-4496-a01b-b06062c04ba9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f4b4a662db049ecacc865ecff10bdb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e30ccf0c1204727b786876c141dbcd2', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:15:32.735154 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5afc883-3ba8-4496-a01b-b06062c04ba9 service nova] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:15:32.735436 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5afc883-3ba8-4496-a01b-b06062c04ba9 service nova] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:15:32.735698 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5afc883-3ba8-4496-a01b-b06062c04ba9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:15:32.737577 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-b5afc883-3ba8-4496-a01b-b06062c04ba9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:15:32.752141 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5afc883-3ba8-4496-a01b-b06062c04ba9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:15:32.754933 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5afc883-3ba8-4496-a01b-b06062c04ba9 service nova] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:15:32.761433 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d5edfa-d20c-435f-9a20-678f952483de None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:15:32.765117 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2018e066-1828-4d3a-b5fb-4963f49c3a9d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91da2acd2534423091537e1ba068d2b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e30ccf0c1204727b786876c141dbcd2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:15:32.765605 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1400/2817] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 11:15:32 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 26 11:15:32.765729 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2018e066-1828-4d3a-b5fb-4963f49c3a9d service neutron] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:15:32.766007 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2018e066-1828-4d3a-b5fb-4963f49c3a9d service neutron] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:15:32.766294 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2018e066-1828-4d3a-b5fb-4963f49c3a9d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:15:32.777014 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2018e066-1828-4d3a-b5fb-4963f49c3a9d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:15:32.778487 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2018e066-1828-4d3a-b5fb-4963f49c3a9d service neutron] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:15:32.789645 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1418/2818] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 11:15:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 26 11:15:33.104285 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5afc883-3ba8-4496-a01b-b06062c04ba9 service nova] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:15:33.105961 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-b5afc883-3ba8-4496-a01b-b06062c04ba9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:15:33.123397 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5afc883-3ba8-4496-a01b-b06062c04ba9 service nova] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:15:33.130215 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754fe001-efb8-4589-8cd0-6839d8b20c42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:15:33.131520 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754fe001-efb8-4589-8cd0-6839d8b20c42 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:15:33.131806 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754fe001-efb8-4589-8cd0-6839d8b20c42 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:15:33.132115 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754fe001-efb8-4589-8cd0-6839d8b20c42 admin admin] PATH_INFO: `/v3/projects/3e93d0cf8cb54b04a4f51b392e6c7783` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:15:33.133428 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-754fe001-efb8-4589-8cd0-6839d8b20c42 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3e93d0cf8cb54b04a4f51b392e6c7783)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:15:33.135627 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-754fe001-efb8-4589-8cd0-6839d8b20c42 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:15:33.204229 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-754fe001-efb8-4589-8cd0-6839d8b20c42 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3e93d0cf8cb54b04a4f51b392e6c7783', 'request_id': 'req-754fe001-efb8-4589-8cd0-6839d8b20c42'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:15:33.205454 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 11:15:33.212728 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-754fe001-efb8-4589-8cd0-6839d8b20c42 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3e93d0cf8cb54b04a4f51b392e6c7783', 'request_id': 'req-754fe001-efb8-4589-8cd0-6839d8b20c42'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:15:33.220868 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-754fe001-efb8-4589-8cd0-6839d8b20c42 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3e93d0cf8cb54b04a4f51b392e6c7783', 'request_id': 'req-754fe001-efb8-4589-8cd0-6839d8b20c42'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:15:33.240929 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-754fe001-efb8-4589-8cd0-6839d8b20c42 admin admin] The token cache is being invalidate because project 3e93d0cf8cb54b04a4f51b392e6c7783 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81423) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 11:15:33.241239 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-754fe001-efb8-4589-8cd0-6839d8b20c42 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:15:33.241865 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-754fe001-efb8-4589-8cd0-6839d8b20c42 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:15:33.243455 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1401/2819] 10.222.0.40 () {58 vars in 1233 bytes} [Fri Aug 26 11:15:33 2022] DELETE /identity/v3/projects/3e93d0cf8cb54b04a4f51b392e6c7783 => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 26 11:15:33.247517 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2018e066-1828-4d3a-b5fb-4963f49c3a9d service neutron] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:15:33.258962 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2018e066-1828-4d3a-b5fb-4963f49c3a9d service neutron] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:15:33.299251 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 11:15:33.322958 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a95d60f-0804-4f5b-b0c3-39116a1b6cf8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:15:33.323596 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a95d60f-0804-4f5b-b0c3-39116a1b6cf8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:15:33.323871 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a95d60f-0804-4f5b-b0c3-39116a1b6cf8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:15:33.324138 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a95d60f-0804-4f5b-b0c3-39116a1b6cf8 admin admin] PATH_INFO: `/v3/users/73c4b6869c7c4e35af0e489b044929da` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:15:33.325271 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a95d60f-0804-4f5b-b0c3-39116a1b6cf8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=73c4b6869c7c4e35af0e489b044929da)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:15:33.327274 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a95d60f-0804-4f5b-b0c3-39116a1b6cf8 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:15:33.376452 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-4a95d60f-0804-4f5b-b0c3-39116a1b6cf8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '73c4b6869c7c4e35af0e489b044929da'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:15:33.377105 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-4a95d60f-0804-4f5b-b0c3-39116a1b6cf8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '73c4b6869c7c4e35af0e489b044929da'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:15:33.387421 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-4a95d60f-0804-4f5b-b0c3-39116a1b6cf8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '73c4b6869c7c4e35af0e489b044929da'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:15:33.388139 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-4a95d60f-0804-4f5b-b0c3-39116a1b6cf8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '73c4b6869c7c4e35af0e489b044929da'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:15:33.400808 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-4a95d60f-0804-4f5b-b0c3-39116a1b6cf8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '73c4b6869c7c4e35af0e489b044929da'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:15:33.413965 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1419/2820] 10.222.0.40 () {58 vars in 1224 bytes} [Fri Aug 26 11:15:33 2022] DELETE /identity/v3/users/73c4b6869c7c4e35af0e489b044929da => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 11:15:33.705390 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754fe001-efb8-4589-8cd0-6839d8b20c42 admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:15:33.707331 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-754fe001-efb8-4589-8cd0-6839d8b20c42 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:15:33.722295 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754fe001-efb8-4589-8cd0-6839d8b20c42 admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:15:33.795647 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde8794e-478b-467a-be39-ce69d4584cbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:15:33.796300 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde8794e-478b-467a-be39-ce69d4584cbc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:15:33.796582 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde8794e-478b-467a-be39-ce69d4584cbc admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:15:33.796852 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde8794e-478b-467a-be39-ce69d4584cbc admin admin] PATH_INFO: `/v3/projects/91496cfd4dbe4c3890a3688f1f54d8e3` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:15:33.798057 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde8794e-478b-467a-be39-ce69d4584cbc admin admin] RBAC: Authorizing `identity:delete_project(project_id=91496cfd4dbe4c3890a3688f1f54d8e3)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:15:33.800571 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde8794e-478b-467a-be39-ce69d4584cbc admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:15:33.859129 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-bde8794e-478b-467a-be39-ce69d4584cbc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '91496cfd4dbe4c3890a3688f1f54d8e3', 'request_id': 'req-bde8794e-478b-467a-be39-ce69d4584cbc'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:15:33.867838 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-bde8794e-478b-467a-be39-ce69d4584cbc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '91496cfd4dbe4c3890a3688f1f54d8e3', 'request_id': 'req-bde8794e-478b-467a-be39-ce69d4584cbc'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:15:33.875908 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-bde8794e-478b-467a-be39-ce69d4584cbc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '91496cfd4dbe4c3890a3688f1f54d8e3', 'request_id': 'req-bde8794e-478b-467a-be39-ce69d4584cbc'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:15:33.890338 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-bde8794e-478b-467a-be39-ce69d4584cbc admin admin] The token cache is being invalidate because project 91496cfd4dbe4c3890a3688f1f54d8e3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81423) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 11:15:33.890627 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-bde8794e-478b-467a-be39-ce69d4584cbc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:15:33.891282 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-bde8794e-478b-467a-be39-ce69d4584cbc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:15:33.893051 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1402/2821] 10.222.0.40 () {58 vars in 1233 bytes} [Fri Aug 26 11:15:33 2022] DELETE /identity/v3/projects/91496cfd4dbe4c3890a3688f1f54d8e3 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 11:15:33.898096 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2700ae7c-d320-4e09-a088-b6d83f2edfcd None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:15:33.898629 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2700ae7c-d320-4e09-a088-b6d83f2edfcd None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:15:33.898629 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2700ae7c-d320-4e09-a088-b6d83f2edfcd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:15:33.921244 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-2700ae7c-d320-4e09-a088-b6d83f2edfcd None None] MFA Rules not processed for user `ca155a48e2024c32af3fd2e19e5fac11`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 11:15:33.922570 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-2700ae7c-d320-4e09-a088-b6d83f2edfcd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:15:33.991213 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1420/2822] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 11:15:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 11:15:33.999112 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648e6b76-7279-45ee-8e67-c1e1795c94aa None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:15:33.999505 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648e6b76-7279-45ee-8e67-c1e1795c94aa None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:15:33.999856 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648e6b76-7279-45ee-8e67-c1e1795c94aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:15:34.021020 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-648e6b76-7279-45ee-8e67-c1e1795c94aa None None] MFA Rules not processed for user `ca155a48e2024c32af3fd2e19e5fac11`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 11:15:34.022192 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-648e6b76-7279-45ee-8e67-c1e1795c94aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:15:34.030409 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1403/2823] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 11:15:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 11:15:34.035697 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2700ae7c-d320-4e09-a088-b6d83f2edfcd None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:15:34.045923 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2700ae7c-d320-4e09-a088-b6d83f2edfcd None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:15:34.051327 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b22c7f-f063-4d58-aa2d-cb4a127d1e5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:15:34.051981 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b22c7f-f063-4d58-aa2d-cb4a127d1e5f admin admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:15:34.052283 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b22c7f-f063-4d58-aa2d-cb4a127d1e5f admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:15:34.052472 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b22c7f-f063-4d58-aa2d-cb4a127d1e5f admin admin] PATH_INFO: `/v3/domains` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:15:34.053126 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b22c7f-f063-4d58-aa2d-cb4a127d1e5f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 11:15:34.053426 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b22c7f-f063-4d58-aa2d-cb4a127d1e5f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:15:34.055077 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b22c7f-f063-4d58-aa2d-cb4a127d1e5f admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:15:34.067502 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1421/2824] 10.222.0.40 () {58 vars in 1153 bytes} [Fri Aug 26 11:15:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 11:15:34.072062 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648e6b76-7279-45ee-8e67-c1e1795c94aa None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:15:34.082789 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648e6b76-7279-45ee-8e67-c1e1795c94aa None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:15:34.087240 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3118d3d3-d93c-461d-ae02-f60ad9b10b73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:15:34.089544 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3118d3d3-d93c-461d-ae02-f60ad9b10b73 admin admin] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:15:34.089544 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3118d3d3-d93c-461d-ae02-f60ad9b10b73 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:15:34.089544 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3118d3d3-d93c-461d-ae02-f60ad9b10b73 admin admin] PATH_INFO: `/v3/projects` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:15:34.089544 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3118d3d3-d93c-461d-ae02-f60ad9b10b73 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:15:34.091286 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3118d3d3-d93c-461d-ae02-f60ad9b10b73 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:15:34.122474 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1404/2825] 10.222.0.40 () {60 vars in 1153 bytes} [Fri Aug 26 11:15:34 2022] POST /identity/v3/projects => generated 381 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 11:15:34.128721 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b22c7f-f063-4d58-aa2d-cb4a127d1e5f admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:15:34.138957 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b22c7f-f063-4d58-aa2d-cb4a127d1e5f admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:15:34.189421 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 11:15:34.206172 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f0ba3d-5747-491c-9056-9f475316ac37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:15:34.206959 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f0ba3d-5747-491c-9056-9f475316ac37 admin admin] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:15:34.207346 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f0ba3d-5747-491c-9056-9f475316ac37 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:15:34.207690 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f0ba3d-5747-491c-9056-9f475316ac37 admin admin] PATH_INFO: `/v3/users` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:15:34.208683 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f0ba3d-5747-491c-9056-9f475316ac37 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:15:34.210674 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f0ba3d-5747-491c-9056-9f475316ac37 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:15:34.238890 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1422/2826] 10.222.0.40 () {60 vars in 1144 bytes} [Fri Aug 26 11:15:34 2022] POST /identity/v3/users => generated 291 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 11:15:34.240513 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3118d3d3-d93c-461d-ae02-f60ad9b10b73 admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:15:34.254284 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3118d3d3-d93c-461d-ae02-f60ad9b10b73 admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:15:34.259331 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6027f7-7887-40fa-8d73-6869b6543012 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:15:34.259988 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6027f7-7887-40fa-8d73-6869b6543012 admin admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:15:34.260198 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6027f7-7887-40fa-8d73-6869b6543012 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:15:34.260471 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6027f7-7887-40fa-8d73-6869b6543012 admin admin] PATH_INFO: `/v3/roles` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:15:34.261122 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c6027f7-7887-40fa-8d73-6869b6543012 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:15:34.262910 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c6027f7-7887-40fa-8d73-6869b6543012 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:15:34.275013 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1405/2827] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 26 11:15:34 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 11:15:34.279146 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f0ba3d-5747-491c-9056-9f475316ac37 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:15:34.293104 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f0ba3d-5747-491c-9056-9f475316ac37 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:15:34.298026 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ed820c-087b-44c5-934d-352b3df606cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:15:34.298655 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ed820c-087b-44c5-934d-352b3df606cc admin admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:15:34.298937 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ed820c-087b-44c5-934d-352b3df606cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:15:34.299207 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ed820c-087b-44c5-934d-352b3df606cc admin admin] PATH_INFO: `/v3/roles` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:15:34.300047 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ed820c-087b-44c5-934d-352b3df606cc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:15:34.302478 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ed820c-087b-44c5-934d-352b3df606cc admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:15:34.320429 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1423/2828] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 26 11:15:34 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 26 11:15:34.324791 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6027f7-7887-40fa-8d73-6869b6543012 admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:15:34.337196 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6027f7-7887-40fa-8d73-6869b6543012 admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:15:34.342191 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f2949b-a54b-4f26-a6c3-037618abbcce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:15:34.342861 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f2949b-a54b-4f26-a6c3-037618abbcce admin admin] REQUEST_METHOD: `PUT` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:15:34.343174 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f2949b-a54b-4f26-a6c3-037618abbcce admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:15:34.343456 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f2949b-a54b-4f26-a6c3-037618abbcce admin admin] PATH_INFO: `/v3/projects/1e5f560db7f44080805767e25b4a76ce/users/0fb7636322bc4e69afad23bd6373805d/roles/23e174ce743b46eabe618c212691da55` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:15:34.365320 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f2949b-a54b-4f26-a6c3-037618abbcce admin admin] RBAC: Authorizing `identity:create_grant(project_id=1e5f560db7f44080805767e25b4a76ce, user_id=0fb7636322bc4e69afad23bd6373805d, role_id=23e174ce743b46eabe618c212691da55)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:15:34.368153 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f2949b-a54b-4f26-a6c3-037618abbcce admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:15:34.382806 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1406/2829] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 26 11:15:34 2022] PUT /identity/v3/projects/1e5f560db7f44080805767e25b4a76ce/users/0fb7636322bc4e69afad23bd6373805d/roles/23e174ce743b46eabe618c212691da55 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 11:15:34.449873 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ddaf70e-3fc2-4788-97f7-3e0c256eb575 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:15:34.450245 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ddaf70e-3fc2-4788-97f7-3e0c256eb575 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:15:34.450444 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ddaf70e-3fc2-4788-97f7-3e0c256eb575 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:15:34.481942 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-5ddaf70e-3fc2-4788-97f7-3e0c256eb575 None None] MFA Rules not processed for user `0fb7636322bc4e69afad23bd6373805d`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 11:15:34.483302 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-5ddaf70e-3fc2-4788-97f7-3e0c256eb575 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:15:34.557247 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1424/2830] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 11:15:34 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 11:15:34.561800 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f2949b-a54b-4f26-a6c3-037618abbcce admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:15:34.568819 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 11:15:34.638327 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f2949b-a54b-4f26-a6c3-037618abbcce admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:15:34.643782 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ddaf70e-3fc2-4788-97f7-3e0c256eb575 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:15:34.643970 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b39624a-5426-4dd7-9c07-059718105ea2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:15:34.644594 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b39624a-5426-4dd7-9c07-059718105ea2 admin admin] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:15:34.644872 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b39624a-5426-4dd7-9c07-059718105ea2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:15:34.645142 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b39624a-5426-4dd7-9c07-059718105ea2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:15:34.645518 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-5ddaf70e-3fc2-4788-97f7-3e0c256eb575 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:15:34.645836 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b39624a-5426-4dd7-9c07-059718105ea2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:15:34.647723 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b39624a-5426-4dd7-9c07-059718105ea2 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:15:34.687422 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1407/2831] 10.222.0.40 () {60 vars in 1153 bytes} [Fri Aug 26 11:15:34 2022] POST /identity/v3/projects => generated 381 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 11:15:34.692377 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b39624a-5426-4dd7-9c07-059718105ea2 admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:15:34.711431 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ddaf70e-3fc2-4788-97f7-3e0c256eb575 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:15:34.761627 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b39624a-5426-4dd7-9c07-059718105ea2 admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:15:34.766561 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4305fda3-85f3-41e1-b65c-67efe3187a5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:15:34.767177 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4305fda3-85f3-41e1-b65c-67efe3187a5a admin admin] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:15:34.767751 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4305fda3-85f3-41e1-b65c-67efe3187a5a admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:15:34.767872 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29937bbf-d91a-4e2d-9212-b6d7c4fccb57 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91da2acd2534423091537e1ba068d2b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e30ccf0c1204727b786876c141dbcd2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:15:34.767999 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4305fda3-85f3-41e1-b65c-67efe3187a5a admin admin] PATH_INFO: `/v3/users` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:15:34.769088 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29937bbf-d91a-4e2d-9212-b6d7c4fccb57 service neutron] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:15:34.769376 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29937bbf-d91a-4e2d-9212-b6d7c4fccb57 service neutron] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:15:34.769597 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4305fda3-85f3-41e1-b65c-67efe3187a5a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:15:34.769710 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29937bbf-d91a-4e2d-9212-b6d7c4fccb57 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:15:34.771613 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4305fda3-85f3-41e1-b65c-67efe3187a5a admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:15:34.771782 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-29937bbf-d91a-4e2d-9212-b6d7c4fccb57 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:15:34.797557 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1408/2832] 10.222.0.40 () {60 vars in 1144 bytes} [Fri Aug 26 11:15:34 2022] POST /identity/v3/users => generated 297 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 26 11:15:34.802098 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4305fda3-85f3-41e1-b65c-67efe3187a5a admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:15:34.803717 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 11:15:34.812399 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4305fda3-85f3-41e1-b65c-67efe3187a5a admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:15:34.818841 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf6f857-15e7-4d37-802f-61dde9de005a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:15:34.819459 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf6f857-15e7-4d37-802f-61dde9de005a admin admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:15:34.819744 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf6f857-15e7-4d37-802f-61dde9de005a admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:15:34.820015 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf6f857-15e7-4d37-802f-61dde9de005a admin admin] PATH_INFO: `/v3/roles` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:15:34.820793 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf6f857-15e7-4d37-802f-61dde9de005a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:15:34.822489 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf6f857-15e7-4d37-802f-61dde9de005a admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:15:34.835742 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1409/2833] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 26 11:15:34 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 26 11:15:34.840017 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf6f857-15e7-4d37-802f-61dde9de005a admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:15:34.848321 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29937bbf-d91a-4e2d-9212-b6d7c4fccb57 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:15:34.849878 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29937bbf-d91a-4e2d-9212-b6d7c4fccb57 service neutron] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:15:34.851492 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf6f857-15e7-4d37-802f-61dde9de005a admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:15:34.857047 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0295df9-28c1-42e3-ac9b-43f615994287 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:15:34.858105 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0295df9-28c1-42e3-ac9b-43f615994287 admin admin] REQUEST_METHOD: `PUT` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:15:34.858105 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0295df9-28c1-42e3-ac9b-43f615994287 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:15:34.858323 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0295df9-28c1-42e3-ac9b-43f615994287 admin admin] PATH_INFO: `/v3/projects/64227c87fb2c40e89303cb36d45bb88a/users/357cc7378854484e930b7fcfcd004816/roles/1f97e4602e2e4d6498913a81f9ae8d6a` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:15:34.860494 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1425/2834] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 11:15:34 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 218 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 26 11:15:34.878490 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0295df9-28c1-42e3-ac9b-43f615994287 admin admin] RBAC: Authorizing `identity:create_grant(project_id=64227c87fb2c40e89303cb36d45bb88a, user_id=357cc7378854484e930b7fcfcd004816, role_id=1f97e4602e2e4d6498913a81f9ae8d6a)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:15:34.881119 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0295df9-28c1-42e3-ac9b-43f615994287 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:15:34.896101 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1410/2835] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 26 11:15:34 2022] PUT /identity/v3/projects/64227c87fb2c40e89303cb36d45bb88a/users/357cc7378854484e930b7fcfcd004816/roles/1f97e4602e2e4d6498913a81f9ae8d6a => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 11:15:34.899928 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29937bbf-d91a-4e2d-9212-b6d7c4fccb57 service neutron] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:15:34.911128 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29937bbf-d91a-4e2d-9212-b6d7c4fccb57 service neutron] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:15:34.981296 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ed0443-3abc-4e4c-857b-42a69bf017c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:15:34.981917 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ed0443-3abc-4e4c-857b-42a69bf017c1 admin admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:15:34.982226 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ed0443-3abc-4e4c-857b-42a69bf017c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:15:34.982493 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ed0443-3abc-4e4c-857b-42a69bf017c1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:15:34.983133 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ed0443-3abc-4e4c-857b-42a69bf017c1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:15:34.984743 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ed0443-3abc-4e4c-857b-42a69bf017c1 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:15:34.997993 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1426/2836] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 26 11:15:34 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 11:15:35.003673 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0295df9-28c1-42e3-ac9b-43f615994287 admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:15:35.015912 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0295df9-28c1-42e3-ac9b-43f615994287 admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:15:35.022376 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28922a6e-a0db-4346-9ef4-7b3445655b14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:15:35.023179 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28922a6e-a0db-4346-9ef4-7b3445655b14 admin admin] REQUEST_METHOD: `PUT` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:15:35.023458 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28922a6e-a0db-4346-9ef4-7b3445655b14 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:15:35.023726 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28922a6e-a0db-4346-9ef4-7b3445655b14 admin admin] PATH_INFO: `/v3/projects/64227c87fb2c40e89303cb36d45bb88a/users/357cc7378854484e930b7fcfcd004816/roles/1f97e4602e2e4d6498913a81f9ae8d6a` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:15:35.025749 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28922a6e-a0db-4346-9ef4-7b3445655b14 admin admin] RBAC: Authorizing `identity:create_grant(project_id=64227c87fb2c40e89303cb36d45bb88a, user_id=357cc7378854484e930b7fcfcd004816, role_id=1f97e4602e2e4d6498913a81f9ae8d6a)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:15:35.028447 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28922a6e-a0db-4346-9ef4-7b3445655b14 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:15:35.042394 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1411/2837] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 26 11:15:35 2022] PUT /identity/v3/projects/64227c87fb2c40e89303cb36d45bb88a/users/357cc7378854484e930b7fcfcd004816/roles/1f97e4602e2e4d6498913a81f9ae8d6a => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 26 11:15:35.053633 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25df004d-abf4-4b86-8051-982030c4210a None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:15:35.053633 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25df004d-abf4-4b86-8051-982030c4210a None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:15:35.054183 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25df004d-abf4-4b86-8051-982030c4210a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:15:35.087764 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-25df004d-abf4-4b86-8051-982030c4210a None None] MFA Rules not processed for user `357cc7378854484e930b7fcfcd004816`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 11:15:35.089023 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-25df004d-abf4-4b86-8051-982030c4210a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:15:35.167297 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1427/2838] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 11:15:35 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 11:15:35.181262 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28922a6e-a0db-4346-9ef4-7b3445655b14 admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:15:35.183912 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-28922a6e-a0db-4346-9ef4-7b3445655b14 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:15:35.227461 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 11:15:35.260427 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28922a6e-a0db-4346-9ef4-7b3445655b14 admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:15:35.264893 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e1d649-006a-49d4-aa9a-4048192f6717 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c277338a7f814cc6b01c5419775031bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e30ccf0c1204727b786876c141dbcd2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:15:35.265515 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e1d649-006a-49d4-aa9a-4048192f6717 service cinder] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:15:35.265794 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e1d649-006a-49d4-aa9a-4048192f6717 service cinder] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:15:35.266192 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e1d649-006a-49d4-aa9a-4048192f6717 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:15:35.277310 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e1d649-006a-49d4-aa9a-4048192f6717 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:15:35.280117 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e1d649-006a-49d4-aa9a-4048192f6717 service cinder] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:15:35.290029 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1412/2839] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 11:15:35 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 26 11:15:35.729272 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25df004d-abf4-4b86-8051-982030c4210a None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:15:35.780133 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 11:15:35.800752 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25df004d-abf4-4b86-8051-982030c4210a None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:15:35.805366 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebcbdeba-6dad-4604-a0ff-6b7042908f44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:15:35.806093 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebcbdeba-6dad-4604-a0ff-6b7042908f44 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:15:35.806371 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebcbdeba-6dad-4604-a0ff-6b7042908f44 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:15:35.806688 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebcbdeba-6dad-4604-a0ff-6b7042908f44 admin admin] PATH_INFO: `/v3/users/0fb7636322bc4e69afad23bd6373805d` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:15:35.808646 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebcbdeba-6dad-4604-a0ff-6b7042908f44 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0fb7636322bc4e69afad23bd6373805d)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:15:35.809843 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebcbdeba-6dad-4604-a0ff-6b7042908f44 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:15:35.870623 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-ebcbdeba-6dad-4604-a0ff-6b7042908f44 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0fb7636322bc4e69afad23bd6373805d'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:15:35.871377 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-ebcbdeba-6dad-4604-a0ff-6b7042908f44 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0fb7636322bc4e69afad23bd6373805d'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:15:35.883082 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-ebcbdeba-6dad-4604-a0ff-6b7042908f44 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0fb7636322bc4e69afad23bd6373805d'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:15:35.883817 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-ebcbdeba-6dad-4604-a0ff-6b7042908f44 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0fb7636322bc4e69afad23bd6373805d'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:15:35.899245 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-ebcbdeba-6dad-4604-a0ff-6b7042908f44 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0fb7636322bc4e69afad23bd6373805d'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:15:35.914585 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1428/2840] 10.222.0.40 () {58 vars in 1224 bytes} [Fri Aug 26 11:15:35 2022] DELETE /identity/v3/users/0fb7636322bc4e69afad23bd6373805d => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 11:15:35.919387 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073d9391-981d-4db5-bc30-e1d18050269e None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:15:35.919929 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073d9391-981d-4db5-bc30-e1d18050269e None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:15:35.919929 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073d9391-981d-4db5-bc30-e1d18050269e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:15:35.940459 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-073d9391-981d-4db5-bc30-e1d18050269e None None] MFA Rules not processed for user `ca155a48e2024c32af3fd2e19e5fac11`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 11:15:35.941729 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-073d9391-981d-4db5-bc30-e1d18050269e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:15:36.013253 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1413/2841] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 11:15:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 11:15:36.024772 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebcbdeba-6dad-4604-a0ff-6b7042908f44 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:15:36.028730 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-ebcbdeba-6dad-4604-a0ff-6b7042908f44 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:15:36.095877 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebcbdeba-6dad-4604-a0ff-6b7042908f44 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:15:36.099815 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75345b7-c621-4f50-8f0d-c8d1352d7225 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91da2acd2534423091537e1ba068d2b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e30ccf0c1204727b786876c141dbcd2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:15:36.100452 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75345b7-c621-4f50-8f0d-c8d1352d7225 service neutron] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:15:36.100727 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75345b7-c621-4f50-8f0d-c8d1352d7225 service neutron] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:15:36.100994 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75345b7-c621-4f50-8f0d-c8d1352d7225 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:15:36.111561 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e75345b7-c621-4f50-8f0d-c8d1352d7225 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:15:36.113084 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e75345b7-c621-4f50-8f0d-c8d1352d7225 service neutron] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:15:36.126237 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1429/2842] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 11:15:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 11:15:36.445287 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073d9391-981d-4db5-bc30-e1d18050269e None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:15:36.447506 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-073d9391-981d-4db5-bc30-e1d18050269e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:15:36.463254 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073d9391-981d-4db5-bc30-e1d18050269e None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:15:36.470674 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1243bc8-888b-4ade-8617-db5ef592b2a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:15:36.471990 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1243bc8-888b-4ade-8617-db5ef592b2a1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:15:36.472335 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1243bc8-888b-4ade-8617-db5ef592b2a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:15:36.473563 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1243bc8-888b-4ade-8617-db5ef592b2a1 admin admin] PATH_INFO: `/v3/projects/1e5f560db7f44080805767e25b4a76ce` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:15:36.474317 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1243bc8-888b-4ade-8617-db5ef592b2a1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1e5f560db7f44080805767e25b4a76ce)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:15:36.476394 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1243bc8-888b-4ade-8617-db5ef592b2a1 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:15:36.549134 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-d1243bc8-888b-4ade-8617-db5ef592b2a1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1e5f560db7f44080805767e25b4a76ce', 'request_id': 'req-d1243bc8-888b-4ade-8617-db5ef592b2a1'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:15:36.556150 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-d1243bc8-888b-4ade-8617-db5ef592b2a1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1e5f560db7f44080805767e25b4a76ce', 'request_id': 'req-d1243bc8-888b-4ade-8617-db5ef592b2a1'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:15:36.562491 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-d1243bc8-888b-4ade-8617-db5ef592b2a1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1e5f560db7f44080805767e25b4a76ce', 'request_id': 'req-d1243bc8-888b-4ade-8617-db5ef592b2a1'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:15:36.579233 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-d1243bc8-888b-4ade-8617-db5ef592b2a1 admin admin] The token cache is being invalidate because project 1e5f560db7f44080805767e25b4a76ce was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81423) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 11:15:36.579523 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-d1243bc8-888b-4ade-8617-db5ef592b2a1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:15:36.580167 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-d1243bc8-888b-4ade-8617-db5ef592b2a1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:15:36.582743 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1414/2843] 10.222.0.40 () {58 vars in 1233 bytes} [Fri Aug 26 11:15:36 2022] DELETE /identity/v3/projects/1e5f560db7f44080805767e25b4a76ce => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 11:15:36.587030 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75345b7-c621-4f50-8f0d-c8d1352d7225 service neutron] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:15:36.599309 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75345b7-c621-4f50-8f0d-c8d1352d7225 service neutron] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:15:36.666938 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68cdc8b2-bccf-4d6d-a9e9-973fc9fac958 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:15:36.667492 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68cdc8b2-bccf-4d6d-a9e9-973fc9fac958 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:15:36.667771 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68cdc8b2-bccf-4d6d-a9e9-973fc9fac958 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:15:36.668040 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68cdc8b2-bccf-4d6d-a9e9-973fc9fac958 admin admin] PATH_INFO: `/v3/users/357cc7378854484e930b7fcfcd004816` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:15:36.669223 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68cdc8b2-bccf-4d6d-a9e9-973fc9fac958 admin admin] RBAC: Authorizing `identity:delete_user(user_id=357cc7378854484e930b7fcfcd004816)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:15:36.671193 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68cdc8b2-bccf-4d6d-a9e9-973fc9fac958 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:15:36.676581 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 11:15:36.733419 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-68cdc8b2-bccf-4d6d-a9e9-973fc9fac958 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '357cc7378854484e930b7fcfcd004816'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:15:36.733419 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-68cdc8b2-bccf-4d6d-a9e9-973fc9fac958 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '357cc7378854484e930b7fcfcd004816'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:15:36.748374 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-68cdc8b2-bccf-4d6d-a9e9-973fc9fac958 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '357cc7378854484e930b7fcfcd004816'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:15:36.749276 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-68cdc8b2-bccf-4d6d-a9e9-973fc9fac958 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '357cc7378854484e930b7fcfcd004816'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:15:36.765092 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-68cdc8b2-bccf-4d6d-a9e9-973fc9fac958 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '357cc7378854484e930b7fcfcd004816'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:15:36.780545 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1430/2844] 10.222.0.40 () {58 vars in 1224 bytes} [Fri Aug 26 11:15:36 2022] DELETE /identity/v3/users/357cc7378854484e930b7fcfcd004816 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 11:15:36.820366 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1243bc8-888b-4ade-8617-db5ef592b2a1 admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:15:36.821965 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-d1243bc8-888b-4ade-8617-db5ef592b2a1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:15:36.903483 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 11:15:36.929997 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1243bc8-888b-4ade-8617-db5ef592b2a1 admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:15:37.018032 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f701ea46-67d1-48c0-91e8-ccc1e4e70576 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f4b4a662db049ecacc865ecff10bdb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e30ccf0c1204727b786876c141dbcd2', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:15:37.019871 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f701ea46-67d1-48c0-91e8-ccc1e4e70576 service nova] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:15:37.020156 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f701ea46-67d1-48c0-91e8-ccc1e4e70576 service nova] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:15:37.020432 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f701ea46-67d1-48c0-91e8-ccc1e4e70576 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:15:37.022585 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-f701ea46-67d1-48c0-91e8-ccc1e4e70576 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:15:37.037141 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f701ea46-67d1-48c0-91e8-ccc1e4e70576 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:15:37.038804 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f701ea46-67d1-48c0-91e8-ccc1e4e70576 service nova] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:15:37.052401 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1415/2845] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 11:15:36 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 232 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 11:15:37.115016 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68cdc8b2-bccf-4d6d-a9e9-973fc9fac958 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:15:37.121781 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-68cdc8b2-bccf-4d6d-a9e9-973fc9fac958 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:15:37.206113 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68cdc8b2-bccf-4d6d-a9e9-973fc9fac958 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:15:37.213488 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12264bca-1ec4-415d-a008-c64c8f82b919 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:15:37.214299 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12264bca-1ec4-415d-a008-c64c8f82b919 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:15:37.214640 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12264bca-1ec4-415d-a008-c64c8f82b919 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:15:37.214973 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12264bca-1ec4-415d-a008-c64c8f82b919 admin admin] PATH_INFO: `/v3/projects/64227c87fb2c40e89303cb36d45bb88a` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:15:37.216255 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12264bca-1ec4-415d-a008-c64c8f82b919 admin admin] RBAC: Authorizing `identity:delete_project(project_id=64227c87fb2c40e89303cb36d45bb88a)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:15:37.218336 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12264bca-1ec4-415d-a008-c64c8f82b919 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:15:37.282609 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-12264bca-1ec4-415d-a008-c64c8f82b919 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '64227c87fb2c40e89303cb36d45bb88a', 'request_id': 'req-12264bca-1ec4-415d-a008-c64c8f82b919'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:15:37.290028 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-12264bca-1ec4-415d-a008-c64c8f82b919 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '64227c87fb2c40e89303cb36d45bb88a', 'request_id': 'req-12264bca-1ec4-415d-a008-c64c8f82b919'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:15:37.297359 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-12264bca-1ec4-415d-a008-c64c8f82b919 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '64227c87fb2c40e89303cb36d45bb88a', 'request_id': 'req-12264bca-1ec4-415d-a008-c64c8f82b919'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:15:37.311301 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-12264bca-1ec4-415d-a008-c64c8f82b919 admin admin] The token cache is being invalidate because project 64227c87fb2c40e89303cb36d45bb88a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81424) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 11:15:37.311557 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-12264bca-1ec4-415d-a008-c64c8f82b919 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:15:37.312212 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-12264bca-1ec4-415d-a008-c64c8f82b919 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:15:37.314037 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1431/2846] 10.222.0.40 () {58 vars in 1233 bytes} [Fri Aug 26 11:15:37 2022] DELETE /identity/v3/projects/64227c87fb2c40e89303cb36d45bb88a => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 11:15:37.320111 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6776452b-1d99-4798-a0e1-626aef31c342 None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:15:37.320402 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6776452b-1d99-4798-a0e1-626aef31c342 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:15:37.320700 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6776452b-1d99-4798-a0e1-626aef31c342 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:15:37.343330 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-6776452b-1d99-4798-a0e1-626aef31c342 None None] MFA Rules not processed for user `ca155a48e2024c32af3fd2e19e5fac11`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 11:15:37.344961 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-6776452b-1d99-4798-a0e1-626aef31c342 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:15:37.431368 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1416/2847] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 11:15:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 11:15:37.440653 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8718e8d7-04c8-45f3-9e7a-5f7a67ea92f4 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:15:37.440653 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8718e8d7-04c8-45f3-9e7a-5f7a67ea92f4 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:15:37.441204 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8718e8d7-04c8-45f3-9e7a-5f7a67ea92f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:15:37.463259 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-8718e8d7-04c8-45f3-9e7a-5f7a67ea92f4 None None] MFA Rules not processed for user `ca155a48e2024c32af3fd2e19e5fac11`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 11:15:37.464519 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-8718e8d7-04c8-45f3-9e7a-5f7a67ea92f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:15:37.473766 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1432/2848] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 11:15:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 11:15:37.477768 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6776452b-1d99-4798-a0e1-626aef31c342 None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:15:37.491691 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6776452b-1d99-4798-a0e1-626aef31c342 None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:15:37.497536 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc34186-048c-482f-adbe-4a99948f9358 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:15:37.497790 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc34186-048c-482f-adbe-4a99948f9358 admin admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:15:37.498091 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc34186-048c-482f-adbe-4a99948f9358 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:15:37.498363 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc34186-048c-482f-adbe-4a99948f9358 admin admin] PATH_INFO: `/v3/domains` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:15:37.499205 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc34186-048c-482f-adbe-4a99948f9358 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 11:15:37.499511 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc34186-048c-482f-adbe-4a99948f9358 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:15:37.501689 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc34186-048c-482f-adbe-4a99948f9358 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:15:37.516033 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1417/2849] 10.222.0.40 () {58 vars in 1153 bytes} [Fri Aug 26 11:15:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 11:15:37.520690 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8718e8d7-04c8-45f3-9e7a-5f7a67ea92f4 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:15:37.532646 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8718e8d7-04c8-45f3-9e7a-5f7a67ea92f4 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:15:37.538417 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a80795-1cd8-437d-ac42-5eccdc12843b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:15:37.539030 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a80795-1cd8-437d-ac42-5eccdc12843b admin admin] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:15:37.539303 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a80795-1cd8-437d-ac42-5eccdc12843b admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:15:37.539573 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a80795-1cd8-437d-ac42-5eccdc12843b admin admin] PATH_INFO: `/v3/projects` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:15:37.540201 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a80795-1cd8-437d-ac42-5eccdc12843b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:15:37.542393 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a80795-1cd8-437d-ac42-5eccdc12843b admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:15:37.547782 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 11:15:37.594034 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1433/2850] 10.222.0.40 () {60 vars in 1153 bytes} [Fri Aug 26 11:15:37 2022] POST /identity/v3/projects => generated 373 bytes in 74 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 11:15:37.597926 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc34186-048c-482f-adbe-4a99948f9358 admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:15:37.613067 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc34186-048c-482f-adbe-4a99948f9358 admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:15:37.695172 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-793bcec4-2750-4d6c-95f7-4a6bc2d04ce9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:15:37.695394 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793bcec4-2750-4d6c-95f7-4a6bc2d04ce9 admin admin] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:15:37.695743 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793bcec4-2750-4d6c-95f7-4a6bc2d04ce9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:15:37.696090 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793bcec4-2750-4d6c-95f7-4a6bc2d04ce9 admin admin] PATH_INFO: `/v3/users` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:15:37.696814 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-793bcec4-2750-4d6c-95f7-4a6bc2d04ce9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:15:37.699376 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-793bcec4-2750-4d6c-95f7-4a6bc2d04ce9 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:15:37.724258 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 11:15:37.733772 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1418/2851] 10.222.0.40 () {60 vars in 1144 bytes} [Fri Aug 26 11:15:37 2022] POST /identity/v3/users => generated 287 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 11:15:37.738517 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a80795-1cd8-437d-ac42-5eccdc12843b admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:15:37.754005 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a80795-1cd8-437d-ac42-5eccdc12843b admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:15:37.770012 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ea100a-a5ae-42f2-9c44-6895d32cc0a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:15:37.770695 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ea100a-a5ae-42f2-9c44-6895d32cc0a7 admin admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:15:37.770974 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ea100a-a5ae-42f2-9c44-6895d32cc0a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:15:37.771242 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ea100a-a5ae-42f2-9c44-6895d32cc0a7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:15:37.771934 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ea100a-a5ae-42f2-9c44-6895d32cc0a7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:15:37.774341 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ea100a-a5ae-42f2-9c44-6895d32cc0a7 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:15:37.787510 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1434/2852] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 26 11:15:37 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 11:15:37.791770 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-793bcec4-2750-4d6c-95f7-4a6bc2d04ce9 admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:15:37.804682 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-793bcec4-2750-4d6c-95f7-4a6bc2d04ce9 admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:15:37.810123 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2f189b-9f95-4c70-97d1-5121a873dd66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:15:37.810998 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2f189b-9f95-4c70-97d1-5121a873dd66 admin admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:15:37.811233 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2f189b-9f95-4c70-97d1-5121a873dd66 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:15:37.811510 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2f189b-9f95-4c70-97d1-5121a873dd66 admin admin] PATH_INFO: `/v3/roles` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:15:37.813589 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b2f189b-9f95-4c70-97d1-5121a873dd66 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:15:37.814146 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b2f189b-9f95-4c70-97d1-5121a873dd66 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:15:37.831255 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1419/2853] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 26 11:15:37 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 11:15:37.835329 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ea100a-a5ae-42f2-9c44-6895d32cc0a7 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:15:37.848789 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ea100a-a5ae-42f2-9c44-6895d32cc0a7 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:15:37.853503 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db45c42-cd44-4c97-9f08-d4910a2589f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:15:37.857795 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db45c42-cd44-4c97-9f08-d4910a2589f2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:15:37.858232 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db45c42-cd44-4c97-9f08-d4910a2589f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:15:37.858581 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db45c42-cd44-4c97-9f08-d4910a2589f2 admin admin] PATH_INFO: `/v3/projects/867d8f2bb90d43a69ba9694663f6cd6d/users/6f5444d933a64a49b429c67a6f6c796d/roles/23e174ce743b46eabe618c212691da55` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:15:37.877431 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db45c42-cd44-4c97-9f08-d4910a2589f2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=867d8f2bb90d43a69ba9694663f6cd6d, user_id=6f5444d933a64a49b429c67a6f6c796d, role_id=23e174ce743b46eabe618c212691da55)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:15:37.881219 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db45c42-cd44-4c97-9f08-d4910a2589f2 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:15:37.898407 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1435/2854] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 26 11:15:37 2022] PUT /identity/v3/projects/867d8f2bb90d43a69ba9694663f6cd6d/users/6f5444d933a64a49b429c67a6f6c796d/roles/23e174ce743b46eabe618c212691da55 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 11:15:37.908843 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd03b5f4-1793-4afe-ab54-9626ebf9e4a9 None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:15:37.909609 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd03b5f4-1793-4afe-ab54-9626ebf9e4a9 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:15:37.910197 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd03b5f4-1793-4afe-ab54-9626ebf9e4a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:15:37.942947 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-fd03b5f4-1793-4afe-ab54-9626ebf9e4a9 None None] MFA Rules not processed for user `6f5444d933a64a49b429c67a6f6c796d`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 11:15:37.944199 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-fd03b5f4-1793-4afe-ab54-9626ebf9e4a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:15:38.033200 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1420/2855] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 11:15:37 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 11:15:38.046366 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db45c42-cd44-4c97-9f08-d4910a2589f2 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:15:38.048217 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-4db45c42-cd44-4c97-9f08-d4910a2589f2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:15:38.113100 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db45c42-cd44-4c97-9f08-d4910a2589f2 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:15:38.119890 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92591a8f-89e7-4343-be20-fe31339c9894 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c277338a7f814cc6b01c5419775031bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e30ccf0c1204727b786876c141dbcd2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:15:38.120527 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92591a8f-89e7-4343-be20-fe31339c9894 service cinder] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:15:38.120809 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92591a8f-89e7-4343-be20-fe31339c9894 service cinder] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:15:38.121089 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92591a8f-89e7-4343-be20-fe31339c9894 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:15:38.130749 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92591a8f-89e7-4343-be20-fe31339c9894 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:15:38.132207 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92591a8f-89e7-4343-be20-fe31339c9894 service cinder] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:15:38.142655 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1436/2856] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 11:15:38 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 11:15:38.166874 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd03b5f4-1793-4afe-ab54-9626ebf9e4a9 None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:15:38.240771 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd03b5f4-1793-4afe-ab54-9626ebf9e4a9 None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:15:38.245469 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71017881-997a-4a11-9277-8e377e5319cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:15:38.246153 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71017881-997a-4a11-9277-8e377e5319cd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:15:38.246435 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71017881-997a-4a11-9277-8e377e5319cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:15:38.246706 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71017881-997a-4a11-9277-8e377e5319cd admin admin] PATH_INFO: `/v3/users/6f5444d933a64a49b429c67a6f6c796d` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:15:38.247865 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71017881-997a-4a11-9277-8e377e5319cd admin admin] RBAC: Authorizing `identity:delete_user(user_id=6f5444d933a64a49b429c67a6f6c796d)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:15:38.249785 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71017881-997a-4a11-9277-8e377e5319cd admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:15:38.301258 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-71017881-997a-4a11-9277-8e377e5319cd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6f5444d933a64a49b429c67a6f6c796d'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:15:38.309051 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-71017881-997a-4a11-9277-8e377e5319cd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6f5444d933a64a49b429c67a6f6c796d'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:15:38.309767 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-71017881-997a-4a11-9277-8e377e5319cd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6f5444d933a64a49b429c67a6f6c796d'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:15:38.322556 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-71017881-997a-4a11-9277-8e377e5319cd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6f5444d933a64a49b429c67a6f6c796d'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:15:38.323250 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-71017881-997a-4a11-9277-8e377e5319cd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6f5444d933a64a49b429c67a6f6c796d'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:15:38.327691 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 11:15:38.339407 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1421/2857] 10.222.0.40 () {58 vars in 1224 bytes} [Fri Aug 26 11:15:38 2022] DELETE /identity/v3/users/6f5444d933a64a49b429c67a6f6c796d => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 11:15:38.344237 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8d1a4a-40a0-4ac6-b7f1-02c5871e76f0 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:15:38.344517 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8d1a4a-40a0-4ac6-b7f1-02c5871e76f0 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:15:38.344790 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8d1a4a-40a0-4ac6-b7f1-02c5871e76f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:15:38.364329 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-ec8d1a4a-40a0-4ac6-b7f1-02c5871e76f0 None None] MFA Rules not processed for user `ca155a48e2024c32af3fd2e19e5fac11`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 11:15:38.365553 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-ec8d1a4a-40a0-4ac6-b7f1-02c5871e76f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:15:38.427119 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1437/2858] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 11:15:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 11:15:38.442377 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71017881-997a-4a11-9277-8e377e5319cd admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:15:38.444046 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-71017881-997a-4a11-9277-8e377e5319cd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:15:38.509768 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71017881-997a-4a11-9277-8e377e5319cd admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:15:38.513737 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3806b3d0-d7dd-4f14-b7f3-a49de1376011 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91da2acd2534423091537e1ba068d2b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e30ccf0c1204727b786876c141dbcd2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:15:38.514704 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3806b3d0-d7dd-4f14-b7f3-a49de1376011 service neutron] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:15:38.514846 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3806b3d0-d7dd-4f14-b7f3-a49de1376011 service neutron] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:15:38.515252 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3806b3d0-d7dd-4f14-b7f3-a49de1376011 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:15:38.526771 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3806b3d0-d7dd-4f14-b7f3-a49de1376011 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:15:38.528214 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3806b3d0-d7dd-4f14-b7f3-a49de1376011 service neutron] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:15:38.541422 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1422/2859] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 11:15:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 11:15:38.859232 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8d1a4a-40a0-4ac6-b7f1-02c5871e76f0 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:15:38.864110 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-ec8d1a4a-40a0-4ac6-b7f1-02c5871e76f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:15:38.884161 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8d1a4a-40a0-4ac6-b7f1-02c5871e76f0 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:15:38.891703 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a155637-2c00-4f43-a110-a9a66b61ac92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:15:38.892766 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a155637-2c00-4f43-a110-a9a66b61ac92 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:15:38.893460 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a155637-2c00-4f43-a110-a9a66b61ac92 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:15:38.893945 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a155637-2c00-4f43-a110-a9a66b61ac92 admin admin] PATH_INFO: `/v3/projects/867d8f2bb90d43a69ba9694663f6cd6d` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:15:38.895897 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a155637-2c00-4f43-a110-a9a66b61ac92 admin admin] RBAC: Authorizing `identity:delete_project(project_id=867d8f2bb90d43a69ba9694663f6cd6d)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:15:38.899572 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a155637-2c00-4f43-a110-a9a66b61ac92 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:15:38.962773 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=94,keystone:INSERT=2,keystone:DELETE=4 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 11:15:38.964438 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-0a155637-2c00-4f43-a110-a9a66b61ac92 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '867d8f2bb90d43a69ba9694663f6cd6d', 'request_id': 'req-0a155637-2c00-4f43-a110-a9a66b61ac92'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:15:38.974071 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-0a155637-2c00-4f43-a110-a9a66b61ac92 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '867d8f2bb90d43a69ba9694663f6cd6d', 'request_id': 'req-0a155637-2c00-4f43-a110-a9a66b61ac92'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:15:38.983142 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-0a155637-2c00-4f43-a110-a9a66b61ac92 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '867d8f2bb90d43a69ba9694663f6cd6d', 'request_id': 'req-0a155637-2c00-4f43-a110-a9a66b61ac92'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:15:38.996982 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-0a155637-2c00-4f43-a110-a9a66b61ac92 admin admin] The token cache is being invalidate because project 867d8f2bb90d43a69ba9694663f6cd6d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81424) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 11:15:38.997323 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-0a155637-2c00-4f43-a110-a9a66b61ac92 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:15:38.997917 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-0a155637-2c00-4f43-a110-a9a66b61ac92 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:15:38.999443 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1438/2860] 10.222.0.40 () {58 vars in 1233 bytes} [Fri Aug 26 11:15:38 2022] DELETE /identity/v3/projects/867d8f2bb90d43a69ba9694663f6cd6d => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 11:15:39.005941 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99804fc1-32b5-4793-bcbf-3fbda223df29 None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:15:39.006552 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99804fc1-32b5-4793-bcbf-3fbda223df29 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:15:39.006552 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99804fc1-32b5-4793-bcbf-3fbda223df29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:15:39.029384 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-99804fc1-32b5-4793-bcbf-3fbda223df29 None None] MFA Rules not processed for user `ca155a48e2024c32af3fd2e19e5fac11`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 11:15:39.030781 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-99804fc1-32b5-4793-bcbf-3fbda223df29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:15:39.099323 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1423/2861] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 11:15:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 11:15:39.107234 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d63d2c-02a7-4af7-9ebd-2acde828e865 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:15:39.107518 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d63d2c-02a7-4af7-9ebd-2acde828e865 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:15:39.107794 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d63d2c-02a7-4af7-9ebd-2acde828e865 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:15:39.132263 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-16d63d2c-02a7-4af7-9ebd-2acde828e865 None None] MFA Rules not processed for user `ca155a48e2024c32af3fd2e19e5fac11`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 11:15:39.133527 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-16d63d2c-02a7-4af7-9ebd-2acde828e865 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:15:39.142233 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1439/2862] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 11:15:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 11:15:39.146681 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99804fc1-32b5-4793-bcbf-3fbda223df29 None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:15:39.156503 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99804fc1-32b5-4793-bcbf-3fbda223df29 None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:15:39.162536 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4674b3f7-ae3e-4041-a003-5ab54a0bf4bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:15:39.163504 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4674b3f7-ae3e-4041-a003-5ab54a0bf4bd admin admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:15:39.163787 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4674b3f7-ae3e-4041-a003-5ab54a0bf4bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:15:39.164058 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4674b3f7-ae3e-4041-a003-5ab54a0bf4bd admin admin] PATH_INFO: `/v3/domains` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:15:39.164708 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4674b3f7-ae3e-4041-a003-5ab54a0bf4bd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 11:15:39.165009 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4674b3f7-ae3e-4041-a003-5ab54a0bf4bd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:15:39.166716 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4674b3f7-ae3e-4041-a003-5ab54a0bf4bd admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:15:39.179336 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1424/2863] 10.222.0.40 () {58 vars in 1153 bytes} [Fri Aug 26 11:15:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 11:15:39.183753 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d63d2c-02a7-4af7-9ebd-2acde828e865 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:15:39.193396 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d63d2c-02a7-4af7-9ebd-2acde828e865 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:15:39.198767 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb6b501-e2bd-4903-9c5e-b86f3d1ca553 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:15:39.199370 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb6b501-e2bd-4903-9c5e-b86f3d1ca553 admin admin] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:15:39.199643 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb6b501-e2bd-4903-9c5e-b86f3d1ca553 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:15:39.199941 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb6b501-e2bd-4903-9c5e-b86f3d1ca553 admin admin] PATH_INFO: `/v3/projects` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:15:39.200570 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb6b501-e2bd-4903-9c5e-b86f3d1ca553 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:15:39.202516 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb6b501-e2bd-4903-9c5e-b86f3d1ca553 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:15:39.239245 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1440/2864] 10.222.0.40 () {60 vars in 1153 bytes} [Fri Aug 26 11:15:39 2022] POST /identity/v3/projects => generated 369 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 11:15:39.244938 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4674b3f7-ae3e-4041-a003-5ab54a0bf4bd admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:15:39.256283 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4674b3f7-ae3e-4041-a003-5ab54a0bf4bd admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:15:39.317234 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60f3a5b-20ca-47df-89c9-51f1660d55f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:15:39.317936 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60f3a5b-20ca-47df-89c9-51f1660d55f9 admin admin] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:15:39.318238 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60f3a5b-20ca-47df-89c9-51f1660d55f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:15:39.318506 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60f3a5b-20ca-47df-89c9-51f1660d55f9 admin admin] PATH_INFO: `/v3/users` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:15:39.319128 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c60f3a5b-20ca-47df-89c9-51f1660d55f9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:15:39.321035 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c60f3a5b-20ca-47df-89c9-51f1660d55f9 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:15:39.348066 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1425/2865] 10.222.0.40 () {60 vars in 1144 bytes} [Fri Aug 26 11:15:39 2022] POST /identity/v3/users => generated 285 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 11:15:39.352601 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb6b501-e2bd-4903-9c5e-b86f3d1ca553 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:15:39.365033 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb6b501-e2bd-4903-9c5e-b86f3d1ca553 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:15:39.370849 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6eaa988-8159-48c8-93a7-5f1a4e94ae25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:15:39.371558 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6eaa988-8159-48c8-93a7-5f1a4e94ae25 admin admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:15:39.371897 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6eaa988-8159-48c8-93a7-5f1a4e94ae25 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:15:39.372232 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6eaa988-8159-48c8-93a7-5f1a4e94ae25 admin admin] PATH_INFO: `/v3/roles` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:15:39.372950 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6eaa988-8159-48c8-93a7-5f1a4e94ae25 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:15:39.374619 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6eaa988-8159-48c8-93a7-5f1a4e94ae25 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:15:39.386316 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1441/2866] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 26 11:15:39 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 26 11:15:39.390631 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60f3a5b-20ca-47df-89c9-51f1660d55f9 admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:15:39.401514 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60f3a5b-20ca-47df-89c9-51f1660d55f9 admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:15:39.406442 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21346481-97f2-40c0-b4e6-58a0776dfe38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:15:39.407054 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21346481-97f2-40c0-b4e6-58a0776dfe38 admin admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:15:39.407334 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21346481-97f2-40c0-b4e6-58a0776dfe38 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:15:39.407640 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21346481-97f2-40c0-b4e6-58a0776dfe38 admin admin] PATH_INFO: `/v3/roles` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:15:39.408290 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21346481-97f2-40c0-b4e6-58a0776dfe38 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:15:39.410633 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21346481-97f2-40c0-b4e6-58a0776dfe38 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:15:39.422045 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1426/2867] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 26 11:15:39 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 11:15:39.431470 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6eaa988-8159-48c8-93a7-5f1a4e94ae25 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:15:39.441901 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6eaa988-8159-48c8-93a7-5f1a4e94ae25 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:15:39.446724 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a07e72b-c117-4534-9bd3-9e8dee48ef65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:15:39.447379 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a07e72b-c117-4534-9bd3-9e8dee48ef65 admin admin] REQUEST_METHOD: `PUT` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:15:39.447661 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a07e72b-c117-4534-9bd3-9e8dee48ef65 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:15:39.447935 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a07e72b-c117-4534-9bd3-9e8dee48ef65 admin admin] PATH_INFO: `/v3/projects/45c8509367794ee589a56257fce59efa/users/819cdc795f4a49eab79bf1f1e3baad18/roles/23e174ce743b46eabe618c212691da55` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:15:39.463887 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a07e72b-c117-4534-9bd3-9e8dee48ef65 admin admin] RBAC: Authorizing `identity:create_grant(project_id=45c8509367794ee589a56257fce59efa, user_id=819cdc795f4a49eab79bf1f1e3baad18, role_id=23e174ce743b46eabe618c212691da55)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:15:39.466667 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a07e72b-c117-4534-9bd3-9e8dee48ef65 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:15:39.481448 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1442/2868] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 26 11:15:39 2022] PUT /identity/v3/projects/45c8509367794ee589a56257fce59efa/users/819cdc795f4a49eab79bf1f1e3baad18/roles/23e174ce743b46eabe618c212691da55 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 11:15:39.493278 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1c36b7-2093-445e-bcd8-28fe8ca4aaf4 None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:15:39.493562 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1c36b7-2093-445e-bcd8-28fe8ca4aaf4 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:15:39.493836 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1c36b7-2093-445e-bcd8-28fe8ca4aaf4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:15:39.502476 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 11:15:39.524605 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-bf1c36b7-2093-445e-bcd8-28fe8ca4aaf4 None None] MFA Rules not processed for user `819cdc795f4a49eab79bf1f1e3baad18`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 11:15:39.525785 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-bf1c36b7-2093-445e-bcd8-28fe8ca4aaf4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:15:39.593183 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1427/2869] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 11:15:39 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 11:15:39.606256 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a07e72b-c117-4534-9bd3-9e8dee48ef65 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:15:39.608048 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-2a07e72b-c117-4534-9bd3-9e8dee48ef65 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:15:39.667424 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a07e72b-c117-4534-9bd3-9e8dee48ef65 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:15:39.671216 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57bc39e-746e-48b6-b2a9-d7662dcf80bb service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8631d7ecfd2849daa8694aecccd22a05', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e30ccf0c1204727b786876c141dbcd2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:15:39.671848 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57bc39e-746e-48b6-b2a9-d7662dcf80bb service glance] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:15:39.672129 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57bc39e-746e-48b6-b2a9-d7662dcf80bb service glance] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:15:39.672405 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57bc39e-746e-48b6-b2a9-d7662dcf80bb service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:15:39.683012 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c57bc39e-746e-48b6-b2a9-d7662dcf80bb service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:15:39.684450 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c57bc39e-746e-48b6-b2a9-d7662dcf80bb service glance] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:15:39.697639 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1443/2870] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 11:15:39 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 11:15:44.468425 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf1c36b7-2093-445e-bcd8-28fe8ca4aaf4 None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:15:44.470040 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-bf1c36b7-2093-445e-bcd8-28fe8ca4aaf4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:15:44.577873 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf1c36b7-2093-445e-bcd8-28fe8ca4aaf4 None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:15:44.585057 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4262c910-cbdd-40a8-bf8d-a0144e0d029e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f4b4a662db049ecacc865ecff10bdb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e30ccf0c1204727b786876c141dbcd2', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:15:44.586047 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4262c910-cbdd-40a8-bf8d-a0144e0d029e service nova] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:15:44.586506 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4262c910-cbdd-40a8-bf8d-a0144e0d029e service nova] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:15:44.586939 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4262c910-cbdd-40a8-bf8d-a0144e0d029e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:15:44.589102 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-4262c910-cbdd-40a8-bf8d-a0144e0d029e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:15:44.604972 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4262c910-cbdd-40a8-bf8d-a0144e0d029e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:15:44.606731 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4262c910-cbdd-40a8-bf8d-a0144e0d029e service nova] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:15:44.620323 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1428/2871] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 11:15:44 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 11:15:49.690121 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=60,keystone:DELETE=3,keystone:INSERT=3 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 11:15:54.615960 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=64 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 11:16:13.297650 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57bc39e-746e-48b6-b2a9-d7662dcf80bb service glance] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:16:13.370014 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57bc39e-746e-48b6-b2a9-d7662dcf80bb service glance] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:16:13.374937 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b950ac-4f5e-4047-838c-9bc79703175e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:16:13.375833 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b950ac-4f5e-4047-838c-9bc79703175e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:16:13.376137 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b950ac-4f5e-4047-838c-9bc79703175e admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:16:13.376407 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b950ac-4f5e-4047-838c-9bc79703175e admin admin] PATH_INFO: `/v3/users/819cdc795f4a49eab79bf1f1e3baad18` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:16:13.377636 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b950ac-4f5e-4047-838c-9bc79703175e admin admin] RBAC: Authorizing `identity:delete_user(user_id=819cdc795f4a49eab79bf1f1e3baad18)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:16:13.379738 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b950ac-4f5e-4047-838c-9bc79703175e admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:16:13.433566 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-99b950ac-4f5e-4047-838c-9bc79703175e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '819cdc795f4a49eab79bf1f1e3baad18'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:16:13.434233 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-99b950ac-4f5e-4047-838c-9bc79703175e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '819cdc795f4a49eab79bf1f1e3baad18'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:16:13.441677 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-99b950ac-4f5e-4047-838c-9bc79703175e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '819cdc795f4a49eab79bf1f1e3baad18'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:16:13.442420 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-99b950ac-4f5e-4047-838c-9bc79703175e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '819cdc795f4a49eab79bf1f1e3baad18'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:16:13.457942 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-99b950ac-4f5e-4047-838c-9bc79703175e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '819cdc795f4a49eab79bf1f1e3baad18'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:16:13.474447 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1444/2872] 10.222.0.40 () {58 vars in 1224 bytes} [Fri Aug 26 11:16:13 2022] DELETE /identity/v3/users/819cdc795f4a49eab79bf1f1e3baad18 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 11:16:13.479534 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20784c2-7b7e-4348-ac19-edc5444d32cc None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:16:13.480024 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20784c2-7b7e-4348-ac19-edc5444d32cc None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:16:13.480145 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20784c2-7b7e-4348-ac19-edc5444d32cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:16:13.500046 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-a20784c2-7b7e-4348-ac19-edc5444d32cc None None] MFA Rules not processed for user `ca155a48e2024c32af3fd2e19e5fac11`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 11:16:13.501534 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-a20784c2-7b7e-4348-ac19-edc5444d32cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:16:13.563950 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1429/2873] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 11:16:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 11:16:13.574151 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b950ac-4f5e-4047-838c-9bc79703175e admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:16:13.575852 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-99b950ac-4f5e-4047-838c-9bc79703175e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:16:13.639676 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b950ac-4f5e-4047-838c-9bc79703175e admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:16:13.643373 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8efa1593-688b-4d6b-8a09-949af3c5b1fe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91da2acd2534423091537e1ba068d2b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e30ccf0c1204727b786876c141dbcd2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:16:13.643990 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efa1593-688b-4d6b-8a09-949af3c5b1fe service neutron] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:16:13.644266 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efa1593-688b-4d6b-8a09-949af3c5b1fe service neutron] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:16:13.644538 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efa1593-688b-4d6b-8a09-949af3c5b1fe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:16:13.654339 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8efa1593-688b-4d6b-8a09-949af3c5b1fe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:16:13.655785 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8efa1593-688b-4d6b-8a09-949af3c5b1fe service neutron] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:16:13.666221 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1445/2874] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 11:16:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 11:16:13.948373 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20784c2-7b7e-4348-ac19-edc5444d32cc None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:16:13.950170 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-a20784c2-7b7e-4348-ac19-edc5444d32cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:16:13.963846 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20784c2-7b7e-4348-ac19-edc5444d32cc None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:16:13.969507 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87bbe8fd-9bb7-4986-8a5c-3d318475b404 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:16:13.970201 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bbe8fd-9bb7-4986-8a5c-3d318475b404 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:16:13.970476 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bbe8fd-9bb7-4986-8a5c-3d318475b404 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:16:13.971457 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bbe8fd-9bb7-4986-8a5c-3d318475b404 admin admin] PATH_INFO: `/v3/projects/45c8509367794ee589a56257fce59efa` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:16:13.972456 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87bbe8fd-9bb7-4986-8a5c-3d318475b404 admin admin] RBAC: Authorizing `identity:delete_project(project_id=45c8509367794ee589a56257fce59efa)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:16:13.976180 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87bbe8fd-9bb7-4986-8a5c-3d318475b404 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:16:14.030065 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-87bbe8fd-9bb7-4986-8a5c-3d318475b404 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '45c8509367794ee589a56257fce59efa', 'request_id': 'req-87bbe8fd-9bb7-4986-8a5c-3d318475b404'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:16:14.036621 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-87bbe8fd-9bb7-4986-8a5c-3d318475b404 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '45c8509367794ee589a56257fce59efa', 'request_id': 'req-87bbe8fd-9bb7-4986-8a5c-3d318475b404'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:16:14.043119 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-87bbe8fd-9bb7-4986-8a5c-3d318475b404 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '45c8509367794ee589a56257fce59efa', 'request_id': 'req-87bbe8fd-9bb7-4986-8a5c-3d318475b404'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:16:14.056001 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-87bbe8fd-9bb7-4986-8a5c-3d318475b404 admin admin] The token cache is being invalidate because project 45c8509367794ee589a56257fce59efa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81423) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 11:16:14.056446 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-87bbe8fd-9bb7-4986-8a5c-3d318475b404 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:16:14.056975 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-87bbe8fd-9bb7-4986-8a5c-3d318475b404 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:16:14.058532 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1430/2875] 10.222.0.40 () {58 vars in 1233 bytes} [Fri Aug 26 11:16:13 2022] DELETE /identity/v3/projects/45c8509367794ee589a56257fce59efa => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 11:16:14.063497 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9e0aca-496c-481b-9c85-59df4e3dd900 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:16:14.063778 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9e0aca-496c-481b-9c85-59df4e3dd900 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:16:14.064074 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9e0aca-496c-481b-9c85-59df4e3dd900 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:16:14.085004 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-8e9e0aca-496c-481b-9c85-59df4e3dd900 None None] MFA Rules not processed for user `ca155a48e2024c32af3fd2e19e5fac11`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 11:16:14.086171 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-8e9e0aca-496c-481b-9c85-59df4e3dd900 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:16:14.104170 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 11:16:14.153657 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1446/2876] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 11:16:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 11:16:14.161712 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f723b75-f91f-4ec1-9579-07d950e77d3b None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:16:14.162562 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f723b75-f91f-4ec1-9579-07d950e77d3b None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:16:14.162803 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f723b75-f91f-4ec1-9579-07d950e77d3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:16:14.183927 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-6f723b75-f91f-4ec1-9579-07d950e77d3b None None] MFA Rules not processed for user `ca155a48e2024c32af3fd2e19e5fac11`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 11:16:14.185157 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-6f723b75-f91f-4ec1-9579-07d950e77d3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:16:14.193265 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1431/2877] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 11:16:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 11:16:14.197923 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9e0aca-496c-481b-9c85-59df4e3dd900 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:16:14.208047 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9e0aca-496c-481b-9c85-59df4e3dd900 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:16:14.212877 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78167b06-0efe-4fed-85be-606d07193321 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:16:14.213492 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78167b06-0efe-4fed-85be-606d07193321 admin admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:16:14.213772 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78167b06-0efe-4fed-85be-606d07193321 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:16:14.214078 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78167b06-0efe-4fed-85be-606d07193321 admin admin] PATH_INFO: `/v3/domains` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:16:14.214733 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78167b06-0efe-4fed-85be-606d07193321 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 11:16:14.215037 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78167b06-0efe-4fed-85be-606d07193321 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:16:14.216646 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78167b06-0efe-4fed-85be-606d07193321 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:16:14.229593 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1447/2878] 10.222.0.40 () {58 vars in 1153 bytes} [Fri Aug 26 11:16:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 11:16:14.234022 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f723b75-f91f-4ec1-9579-07d950e77d3b None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:16:14.245287 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f723b75-f91f-4ec1-9579-07d950e77d3b None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:16:14.250029 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62b8a4b-7ba9-4d91-afce-d733abe7d98b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:16:14.250883 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62b8a4b-7ba9-4d91-afce-d733abe7d98b admin admin] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:16:14.251156 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62b8a4b-7ba9-4d91-afce-d733abe7d98b admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:16:14.251421 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62b8a4b-7ba9-4d91-afce-d733abe7d98b admin admin] PATH_INFO: `/v3/projects` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:16:14.252038 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c62b8a4b-7ba9-4d91-afce-d733abe7d98b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:16:14.253912 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c62b8a4b-7ba9-4d91-afce-d733abe7d98b admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:16:14.280635 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1432/2879] 10.222.0.40 () {60 vars in 1153 bytes} [Fri Aug 26 11:16:14 2022] POST /identity/v3/projects => generated 371 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 11:16:14.284940 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78167b06-0efe-4fed-85be-606d07193321 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:16:14.296189 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78167b06-0efe-4fed-85be-606d07193321 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:16:14.358300 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8616536f-489f-4f22-8a71-da3dc3963386 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:16:14.359244 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8616536f-489f-4f22-8a71-da3dc3963386 admin admin] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:16:14.359734 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8616536f-489f-4f22-8a71-da3dc3963386 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:16:14.360193 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8616536f-489f-4f22-8a71-da3dc3963386 admin admin] PATH_INFO: `/v3/users` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:16:14.361264 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8616536f-489f-4f22-8a71-da3dc3963386 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:16:14.363641 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8616536f-489f-4f22-8a71-da3dc3963386 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:16:14.386811 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1448/2880] 10.222.0.40 () {60 vars in 1144 bytes} [Fri Aug 26 11:16:14 2022] POST /identity/v3/users => generated 286 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 11:16:14.391160 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62b8a4b-7ba9-4d91-afce-d733abe7d98b admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:16:14.402348 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62b8a4b-7ba9-4d91-afce-d733abe7d98b admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:16:14.407205 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec1dde3-ad19-4b78-8a62-78f0e12185bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:16:14.407814 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec1dde3-ad19-4b78-8a62-78f0e12185bc admin admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:16:14.408093 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec1dde3-ad19-4b78-8a62-78f0e12185bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:16:14.408373 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec1dde3-ad19-4b78-8a62-78f0e12185bc admin admin] PATH_INFO: `/v3/roles` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:16:14.409023 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec1dde3-ad19-4b78-8a62-78f0e12185bc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:16:14.410667 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec1dde3-ad19-4b78-8a62-78f0e12185bc admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:16:14.422082 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1433/2881] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 26 11:16:14 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 11:16:14.426353 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8616536f-489f-4f22-8a71-da3dc3963386 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:16:14.437307 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8616536f-489f-4f22-8a71-da3dc3963386 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:16:14.442174 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d21696-3fe0-40d0-a45d-7e8afc7c9fce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:16:14.442779 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d21696-3fe0-40d0-a45d-7e8afc7c9fce admin admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:16:14.443089 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d21696-3fe0-40d0-a45d-7e8afc7c9fce admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:16:14.443409 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d21696-3fe0-40d0-a45d-7e8afc7c9fce admin admin] PATH_INFO: `/v3/roles` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:16:14.444308 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d21696-3fe0-40d0-a45d-7e8afc7c9fce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:16:14.445966 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d21696-3fe0-40d0-a45d-7e8afc7c9fce admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:16:14.457317 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1449/2882] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 26 11:16:14 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 11:16:14.462523 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec1dde3-ad19-4b78-8a62-78f0e12185bc admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:16:14.472653 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec1dde3-ad19-4b78-8a62-78f0e12185bc admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:16:14.478576 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d8760c-d7b0-4701-90e2-adf7bbae0ae6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:16:14.479268 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d8760c-d7b0-4701-90e2-adf7bbae0ae6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:16:14.479541 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d8760c-d7b0-4701-90e2-adf7bbae0ae6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:16:14.479809 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d8760c-d7b0-4701-90e2-adf7bbae0ae6 admin admin] PATH_INFO: `/v3/projects/eb35e2745c404e38b13dfacbef2b2e4e/users/9aa4e563bc62406cbd5a821ef445650d/roles/23e174ce743b46eabe618c212691da55` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:16:14.496839 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d8760c-d7b0-4701-90e2-adf7bbae0ae6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=eb35e2745c404e38b13dfacbef2b2e4e, user_id=9aa4e563bc62406cbd5a821ef445650d, role_id=23e174ce743b46eabe618c212691da55)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:16:14.499551 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d8760c-d7b0-4701-90e2-adf7bbae0ae6 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:16:14.514022 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1434/2883] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 26 11:16:14 2022] PUT /identity/v3/projects/eb35e2745c404e38b13dfacbef2b2e4e/users/9aa4e563bc62406cbd5a821ef445650d/roles/23e174ce743b46eabe618c212691da55 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 11:16:14.527515 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7212385e-994f-4190-b029-13c823aae21a None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:16:14.528012 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7212385e-994f-4190-b029-13c823aae21a None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:16:14.528484 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7212385e-994f-4190-b029-13c823aae21a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:16:14.561635 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-7212385e-994f-4190-b029-13c823aae21a None None] MFA Rules not processed for user `9aa4e563bc62406cbd5a821ef445650d`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 11:16:14.562888 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-7212385e-994f-4190-b029-13c823aae21a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:16:14.627172 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1450/2884] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 11:16:14 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 11:16:14.638688 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d8760c-d7b0-4701-90e2-adf7bbae0ae6 admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:16:14.640411 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-95d8760c-d7b0-4701-90e2-adf7bbae0ae6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:16:14.663890 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 11:16:14.702422 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d8760c-d7b0-4701-90e2-adf7bbae0ae6 admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:16:14.706027 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645c2e24-f053-4cd2-8b75-7f824855b906 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c277338a7f814cc6b01c5419775031bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e30ccf0c1204727b786876c141dbcd2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:16:14.706628 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645c2e24-f053-4cd2-8b75-7f824855b906 service cinder] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:16:14.706903 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645c2e24-f053-4cd2-8b75-7f824855b906 service cinder] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:16:14.707171 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645c2e24-f053-4cd2-8b75-7f824855b906 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:16:14.717341 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-645c2e24-f053-4cd2-8b75-7f824855b906 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:16:14.718829 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-645c2e24-f053-4cd2-8b75-7f824855b906 service cinder] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:16:14.730302 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1435/2885] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 11:16:14 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 11:16:14.759141 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7212385e-994f-4190-b029-13c823aae21a None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:16:14.778742 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 11:16:14.825284 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7212385e-994f-4190-b029-13c823aae21a None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:16:14.829887 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf18d0f-44fa-4e30-b613-61a82f4e7f4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:16:14.830557 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf18d0f-44fa-4e30-b613-61a82f4e7f4c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:16:14.830836 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf18d0f-44fa-4e30-b613-61a82f4e7f4c admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:16:14.831107 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf18d0f-44fa-4e30-b613-61a82f4e7f4c admin admin] PATH_INFO: `/v3/users/9aa4e563bc62406cbd5a821ef445650d` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:16:14.832254 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf18d0f-44fa-4e30-b613-61a82f4e7f4c admin admin] RBAC: Authorizing `identity:delete_user(user_id=9aa4e563bc62406cbd5a821ef445650d)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:16:14.834444 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf18d0f-44fa-4e30-b613-61a82f4e7f4c admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:16:14.885152 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-caf18d0f-44fa-4e30-b613-61a82f4e7f4c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9aa4e563bc62406cbd5a821ef445650d'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:16:14.885834 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-caf18d0f-44fa-4e30-b613-61a82f4e7f4c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9aa4e563bc62406cbd5a821ef445650d'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:16:14.892822 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-caf18d0f-44fa-4e30-b613-61a82f4e7f4c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9aa4e563bc62406cbd5a821ef445650d'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:16:14.893473 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-caf18d0f-44fa-4e30-b613-61a82f4e7f4c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9aa4e563bc62406cbd5a821ef445650d'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:16:14.905962 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-caf18d0f-44fa-4e30-b613-61a82f4e7f4c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9aa4e563bc62406cbd5a821ef445650d'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:16:14.919861 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1451/2886] 10.222.0.40 () {58 vars in 1224 bytes} [Fri Aug 26 11:16:14 2022] DELETE /identity/v3/users/9aa4e563bc62406cbd5a821ef445650d => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 26 11:16:14.924473 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1550b465-bb61-4e35-a2ab-cde3d3e72295 None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:16:14.924782 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1550b465-bb61-4e35-a2ab-cde3d3e72295 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:16:14.925057 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1550b465-bb61-4e35-a2ab-cde3d3e72295 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:16:14.944805 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-1550b465-bb61-4e35-a2ab-cde3d3e72295 None None] MFA Rules not processed for user `ca155a48e2024c32af3fd2e19e5fac11`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 11:16:14.946046 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-1550b465-bb61-4e35-a2ab-cde3d3e72295 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:16:15.005778 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1436/2887] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 11:16:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 11:16:15.015853 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf18d0f-44fa-4e30-b613-61a82f4e7f4c admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:16:15.017270 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-caf18d0f-44fa-4e30-b613-61a82f4e7f4c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:16:15.079919 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf18d0f-44fa-4e30-b613-61a82f4e7f4c admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:16:15.083643 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c063108-70c3-40ee-8abe-d43751cd9ed2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91da2acd2534423091537e1ba068d2b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e30ccf0c1204727b786876c141dbcd2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:16:15.085124 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c063108-70c3-40ee-8abe-d43751cd9ed2 service neutron] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:16:15.085512 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c063108-70c3-40ee-8abe-d43751cd9ed2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:16:15.085780 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c063108-70c3-40ee-8abe-d43751cd9ed2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:16:15.095428 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c063108-70c3-40ee-8abe-d43751cd9ed2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:16:15.096864 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c063108-70c3-40ee-8abe-d43751cd9ed2 service neutron] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:16:15.107515 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1452/2888] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 11:16:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 11:16:15.397731 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1550b465-bb61-4e35-a2ab-cde3d3e72295 None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:16:15.399219 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-1550b465-bb61-4e35-a2ab-cde3d3e72295 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:16:15.412895 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1550b465-bb61-4e35-a2ab-cde3d3e72295 None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:16:15.417734 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7e02e1-f7e7-4942-93d0-1681ee5971b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:16:15.418429 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7e02e1-f7e7-4942-93d0-1681ee5971b1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:16:15.418687 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7e02e1-f7e7-4942-93d0-1681ee5971b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:16:15.418958 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7e02e1-f7e7-4942-93d0-1681ee5971b1 admin admin] PATH_INFO: `/v3/projects/eb35e2745c404e38b13dfacbef2b2e4e` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:16:15.420097 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb7e02e1-f7e7-4942-93d0-1681ee5971b1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=eb35e2745c404e38b13dfacbef2b2e4e)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:16:15.422033 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb7e02e1-f7e7-4942-93d0-1681ee5971b1 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:16:15.473875 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-bb7e02e1-f7e7-4942-93d0-1681ee5971b1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eb35e2745c404e38b13dfacbef2b2e4e', 'request_id': 'req-bb7e02e1-f7e7-4942-93d0-1681ee5971b1'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:16:15.481575 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-bb7e02e1-f7e7-4942-93d0-1681ee5971b1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eb35e2745c404e38b13dfacbef2b2e4e', 'request_id': 'req-bb7e02e1-f7e7-4942-93d0-1681ee5971b1'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:16:15.487250 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-bb7e02e1-f7e7-4942-93d0-1681ee5971b1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eb35e2745c404e38b13dfacbef2b2e4e', 'request_id': 'req-bb7e02e1-f7e7-4942-93d0-1681ee5971b1'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:16:15.501536 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-bb7e02e1-f7e7-4942-93d0-1681ee5971b1 admin admin] The token cache is being invalidate because project eb35e2745c404e38b13dfacbef2b2e4e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81423) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 11:16:15.501820 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-bb7e02e1-f7e7-4942-93d0-1681ee5971b1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:16:15.502511 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-bb7e02e1-f7e7-4942-93d0-1681ee5971b1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:16:15.504060 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1437/2889] 10.222.0.40 () {58 vars in 1233 bytes} [Fri Aug 26 11:16:15 2022] DELETE /identity/v3/projects/eb35e2745c404e38b13dfacbef2b2e4e => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 11:16:25.103004 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=71,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 11:16:25.497997 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=71,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 11:17:49.997878 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c063108-70c3-40ee-8abe-d43751cd9ed2 service neutron] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:17:50.006821 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c063108-70c3-40ee-8abe-d43751cd9ed2 service neutron] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:17:50.056460 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4f68fb-d6a6-404c-a65a-d4b177a1a2be service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91da2acd2534423091537e1ba068d2b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e30ccf0c1204727b786876c141dbcd2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:17:50.057060 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4f68fb-d6a6-404c-a65a-d4b177a1a2be service neutron] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:17:50.057332 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4f68fb-d6a6-404c-a65a-d4b177a1a2be service neutron] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:17:50.057598 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4f68fb-d6a6-404c-a65a-d4b177a1a2be service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:17:50.059701 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-fa4f68fb-d6a6-404c-a65a-d4b177a1a2be service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:17:50.124630 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa4f68fb-d6a6-404c-a65a-d4b177a1a2be service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:17:50.126354 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa4f68fb-d6a6-404c-a65a-d4b177a1a2be service neutron] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:17:50.136567 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1453/2890] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 11:17:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 11:17:55.912225 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7e02e1-f7e7-4942-93d0-1681ee5971b1 admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:17:55.914617 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-bb7e02e1-f7e7-4942-93d0-1681ee5971b1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:17:56.006056 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7e02e1-f7e7-4942-93d0-1681ee5971b1 admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:17:56.011953 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1603da4-0530-4a33-a860-d94572743229 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f29c5dfed31f42a9b5cbc0cd72e9fdc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e30ccf0c1204727b786876c141dbcd2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:17:56.012649 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1603da4-0530-4a33-a860-d94572743229 service placement] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:17:56.013038 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1603da4-0530-4a33-a860-d94572743229 service placement] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:17:56.013831 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1603da4-0530-4a33-a860-d94572743229 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:17:56.015565 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-d1603da4-0530-4a33-a860-d94572743229 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:17:56.028391 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1603da4-0530-4a33-a860-d94572743229 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:17:56.029854 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1603da4-0530-4a33-a860-d94572743229 service placement] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:17:56.042155 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1438/2891] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 11:17:55 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 11:17:56.870640 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4f68fb-d6a6-404c-a65a-d4b177a1a2be service neutron] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:17:56.872849 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-fa4f68fb-d6a6-404c-a65a-d4b177a1a2be service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:17:56.887318 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4f68fb-d6a6-404c-a65a-d4b177a1a2be service neutron] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:17:56.898004 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e025f3-9f85-4b4a-825e-4d919e9fc02c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:17:56.898873 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e025f3-9f85-4b4a-825e-4d919e9fc02c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:17:56.899545 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e025f3-9f85-4b4a-825e-4d919e9fc02c admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:17:56.900159 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e025f3-9f85-4b4a-825e-4d919e9fc02c admin admin] PATH_INFO: `/v3/users/9e8355dc324e49d881b9f0fc5c2923ad` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:17:56.902257 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e025f3-9f85-4b4a-825e-4d919e9fc02c admin admin] RBAC: Authorizing `identity:delete_user(user_id=9e8355dc324e49d881b9f0fc5c2923ad)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:17:56.904232 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e025f3-9f85-4b4a-825e-4d919e9fc02c admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:17:56.961874 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-01e025f3-9f85-4b4a-825e-4d919e9fc02c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9e8355dc324e49d881b9f0fc5c2923ad'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:17:56.962588 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-01e025f3-9f85-4b4a-825e-4d919e9fc02c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9e8355dc324e49d881b9f0fc5c2923ad'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:17:56.970228 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-01e025f3-9f85-4b4a-825e-4d919e9fc02c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9e8355dc324e49d881b9f0fc5c2923ad'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:17:56.970958 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-01e025f3-9f85-4b4a-825e-4d919e9fc02c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9e8355dc324e49d881b9f0fc5c2923ad'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:17:56.984389 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-01e025f3-9f85-4b4a-825e-4d919e9fc02c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9e8355dc324e49d881b9f0fc5c2923ad'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:17:56.998260 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1454/2892] 10.222.0.40 () {58 vars in 1224 bytes} [Fri Aug 26 11:17:56 2022] DELETE /identity/v3/users/9e8355dc324e49d881b9f0fc5c2923ad => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 11:17:57.176151 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1603da4-0530-4a33-a860-d94572743229 service placement] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:17:57.187190 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1603da4-0530-4a33-a860-d94572743229 service placement] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:17:57.249901 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302c2d66-7053-4b24-8465-a952c5e89d58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:17:57.250581 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302c2d66-7053-4b24-8465-a952c5e89d58 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:17:57.250898 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302c2d66-7053-4b24-8465-a952c5e89d58 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:17:57.251503 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302c2d66-7053-4b24-8465-a952c5e89d58 admin admin] PATH_INFO: `/v3/projects/d773f20240a44729b159c57c80e017ef` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:17:57.252654 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-302c2d66-7053-4b24-8465-a952c5e89d58 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d773f20240a44729b159c57c80e017ef)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:17:57.254602 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-302c2d66-7053-4b24-8465-a952c5e89d58 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:17:57.301809 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-302c2d66-7053-4b24-8465-a952c5e89d58 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd773f20240a44729b159c57c80e017ef', 'request_id': 'req-302c2d66-7053-4b24-8465-a952c5e89d58'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:17:57.309207 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-302c2d66-7053-4b24-8465-a952c5e89d58 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd773f20240a44729b159c57c80e017ef', 'request_id': 'req-302c2d66-7053-4b24-8465-a952c5e89d58'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:17:57.314823 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-302c2d66-7053-4b24-8465-a952c5e89d58 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd773f20240a44729b159c57c80e017ef', 'request_id': 'req-302c2d66-7053-4b24-8465-a952c5e89d58'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:17:57.327078 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-302c2d66-7053-4b24-8465-a952c5e89d58 admin admin] The token cache is being invalidate because project d773f20240a44729b159c57c80e017ef was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81423) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 11:17:57.327360 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-302c2d66-7053-4b24-8465-a952c5e89d58 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:17:57.327979 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-302c2d66-7053-4b24-8465-a952c5e89d58 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:17:57.329449 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1439/2893] 10.222.0.40 () {58 vars in 1233 bytes} [Fri Aug 26 11:17:57 2022] DELETE /identity/v3/projects/d773f20240a44729b159c57c80e017ef => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 11:17:57.333447 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e025f3-9f85-4b4a-825e-4d919e9fc02c admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:17:57.334896 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-01e025f3-9f85-4b4a-825e-4d919e9fc02c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:17:57.349296 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e025f3-9f85-4b4a-825e-4d919e9fc02c admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:17:57.367849 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 11:17:57.410614 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1a61df-8ef5-4c4a-9238-1a94d6fd75c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:17:57.411250 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1a61df-8ef5-4c4a-9238-1a94d6fd75c5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:17:57.411533 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1a61df-8ef5-4c4a-9238-1a94d6fd75c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:17:57.411804 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1a61df-8ef5-4c4a-9238-1a94d6fd75c5 admin admin] PATH_INFO: `/v3/users/379400ce9edd4388b57f43664a2d62c7` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:17:57.412948 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c1a61df-8ef5-4c4a-9238-1a94d6fd75c5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=379400ce9edd4388b57f43664a2d62c7)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:17:57.414910 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c1a61df-8ef5-4c4a-9238-1a94d6fd75c5 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:17:57.461336 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-3c1a61df-8ef5-4c4a-9238-1a94d6fd75c5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '379400ce9edd4388b57f43664a2d62c7'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:17:57.462040 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-3c1a61df-8ef5-4c4a-9238-1a94d6fd75c5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '379400ce9edd4388b57f43664a2d62c7'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:17:57.469469 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-3c1a61df-8ef5-4c4a-9238-1a94d6fd75c5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '379400ce9edd4388b57f43664a2d62c7'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:17:57.470195 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-3c1a61df-8ef5-4c4a-9238-1a94d6fd75c5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '379400ce9edd4388b57f43664a2d62c7'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:17:57.482613 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-3c1a61df-8ef5-4c4a-9238-1a94d6fd75c5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '379400ce9edd4388b57f43664a2d62c7'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:17:57.494963 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1455/2894] 10.222.0.40 () {58 vars in 1224 bytes} [Fri Aug 26 11:17:57 2022] DELETE /identity/v3/users/379400ce9edd4388b57f43664a2d62c7 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 11:17:57.665153 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302c2d66-7053-4b24-8465-a952c5e89d58 admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:17:57.666700 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-302c2d66-7053-4b24-8465-a952c5e89d58 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:17:57.683667 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302c2d66-7053-4b24-8465-a952c5e89d58 admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:17:57.701524 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 11:17:57.743229 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40250816-5434-47f4-8732-ebbf4f95c3da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:17:57.743858 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40250816-5434-47f4-8732-ebbf4f95c3da admin admin] REQUEST_METHOD: `DELETE` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:17:57.744137 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40250816-5434-47f4-8732-ebbf4f95c3da admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:17:57.744412 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40250816-5434-47f4-8732-ebbf4f95c3da admin admin] PATH_INFO: `/v3/projects/e187724eb241446f840cb3580c714abf` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:17:57.745709 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40250816-5434-47f4-8732-ebbf4f95c3da admin admin] RBAC: Authorizing `identity:delete_project(project_id=e187724eb241446f840cb3580c714abf)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:17:57.747648 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40250816-5434-47f4-8732-ebbf4f95c3da admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:17:57.798336 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-40250816-5434-47f4-8732-ebbf4f95c3da admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e187724eb241446f840cb3580c714abf', 'request_id': 'req-40250816-5434-47f4-8732-ebbf4f95c3da'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:17:57.804544 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-40250816-5434-47f4-8732-ebbf4f95c3da admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e187724eb241446f840cb3580c714abf', 'request_id': 'req-40250816-5434-47f4-8732-ebbf4f95c3da'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:17:57.810802 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-40250816-5434-47f4-8732-ebbf4f95c3da admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e187724eb241446f840cb3580c714abf', 'request_id': 'req-40250816-5434-47f4-8732-ebbf4f95c3da'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:17:57.823063 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-40250816-5434-47f4-8732-ebbf4f95c3da admin admin] The token cache is being invalidate because project e187724eb241446f840cb3580c714abf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81423) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 11:17:57.823926 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-40250816-5434-47f4-8732-ebbf4f95c3da admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:17:57.824546 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-40250816-5434-47f4-8732-ebbf4f95c3da admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:17:57.826061 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1440/2895] 10.222.0.40 () {58 vars in 1233 bytes} [Fri Aug 26 11:17:57 2022] DELETE /identity/v3/projects/e187724eb241446f840cb3580c714abf => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 11:18:00.529857 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1a61df-8ef5-4c4a-9238-1a94d6fd75c5 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:18:00.531419 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-3c1a61df-8ef5-4c4a-9238-1a94d6fd75c5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:18:00.590085 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1a61df-8ef5-4c4a-9238-1a94d6fd75c5 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:18:00.642926 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50234866-83c2-43a2-a152-dc24d375262a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91da2acd2534423091537e1ba068d2b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e30ccf0c1204727b786876c141dbcd2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:18:00.643531 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50234866-83c2-43a2-a152-dc24d375262a service neutron] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:18:00.643810 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50234866-83c2-43a2-a152-dc24d375262a service neutron] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:18:00.644079 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50234866-83c2-43a2-a152-dc24d375262a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:18:00.645845 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-50234866-83c2-43a2-a152-dc24d375262a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:18:00.655959 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50234866-83c2-43a2-a152-dc24d375262a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:18:00.657389 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50234866-83c2-43a2-a152-dc24d375262a service neutron] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:18:00.661447 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 11:18:00.665472 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1456/2896] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 11:18:00 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 11:18:00.786383 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40250816-5434-47f4-8732-ebbf4f95c3da admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:18:00.787950 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-40250816-5434-47f4-8732-ebbf4f95c3da admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:18:00.801195 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40250816-5434-47f4-8732-ebbf4f95c3da admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:18:00.804815 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f727e1-89ac-44f3-8f4f-cdc0defb15db service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91da2acd2534423091537e1ba068d2b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e30ccf0c1204727b786876c141dbcd2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:18:00.805413 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f727e1-89ac-44f3-8f4f-cdc0defb15db service neutron] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:18:00.805694 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f727e1-89ac-44f3-8f4f-cdc0defb15db service neutron] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:18:00.805962 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f727e1-89ac-44f3-8f4f-cdc0defb15db service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:18:00.808598 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-e7f727e1-89ac-44f3-8f4f-cdc0defb15db service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:18:00.818454 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f727e1-89ac-44f3-8f4f-cdc0defb15db service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:18:00.819888 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f727e1-89ac-44f3-8f4f-cdc0defb15db service neutron] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:18:00.828839 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1441/2897] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 11:18:00 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 11:18:01.952855 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50234866-83c2-43a2-a152-dc24d375262a service neutron] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:18:01.962147 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50234866-83c2-43a2-a152-dc24d375262a service neutron] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:18:01.966463 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d4042a-4993-4885-b0a8-47a16e5a1c7a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91da2acd2534423091537e1ba068d2b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e30ccf0c1204727b786876c141dbcd2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:18:01.967056 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d4042a-4993-4885-b0a8-47a16e5a1c7a service neutron] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:18:01.967331 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d4042a-4993-4885-b0a8-47a16e5a1c7a service neutron] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:18:01.967600 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d4042a-4993-4885-b0a8-47a16e5a1c7a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:18:01.969301 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-96d4042a-4993-4885-b0a8-47a16e5a1c7a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:18:02.055541 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d4042a-4993-4885-b0a8-47a16e5a1c7a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:18:02.056979 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d4042a-4993-4885-b0a8-47a16e5a1c7a service neutron] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:18:02.068795 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1457/2898] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 11:18:01 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 11:18:10.825075 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=48,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 11:18:12.063914 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=34 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 11:19:46.801959 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f727e1-89ac-44f3-8f4f-cdc0defb15db service neutron] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:19:46.803754 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-e7f727e1-89ac-44f3-8f4f-cdc0defb15db service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:19:46.919151 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f727e1-89ac-44f3-8f4f-cdc0defb15db service neutron] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:19:46.924936 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aab07b2-eaaa-491c-9ade-0ed5abc3f1dd service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f29c5dfed31f42a9b5cbc0cd72e9fdc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e30ccf0c1204727b786876c141dbcd2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:19:46.925635 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aab07b2-eaaa-491c-9ade-0ed5abc3f1dd service placement] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:19:46.926010 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aab07b2-eaaa-491c-9ade-0ed5abc3f1dd service placement] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:19:46.926364 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aab07b2-eaaa-491c-9ade-0ed5abc3f1dd service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:19:46.928189 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-6aab07b2-eaaa-491c-9ade-0ed5abc3f1dd service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:19:46.943091 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aab07b2-eaaa-491c-9ade-0ed5abc3f1dd service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:19:46.944685 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aab07b2-eaaa-491c-9ade-0ed5abc3f1dd service placement] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:19:46.955486 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1442/2899] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 11:19:46 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 11:19:49.296550 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d4042a-4993-4885-b0a8-47a16e5a1c7a service neutron] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:19:49.298014 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-96d4042a-4993-4885-b0a8-47a16e5a1c7a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:19:49.309447 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d4042a-4993-4885-b0a8-47a16e5a1c7a service neutron] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:19:49.313208 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10739778-6477-4238-9c5f-fa768f245985 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91da2acd2534423091537e1ba068d2b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e30ccf0c1204727b786876c141dbcd2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:19:49.313807 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10739778-6477-4238-9c5f-fa768f245985 service neutron] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:19:49.314112 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10739778-6477-4238-9c5f-fa768f245985 service neutron] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:19:49.314388 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10739778-6477-4238-9c5f-fa768f245985 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:19:49.316710 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-10739778-6477-4238-9c5f-fa768f245985 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:19:49.392824 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10739778-6477-4238-9c5f-fa768f245985 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:19:49.394295 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10739778-6477-4238-9c5f-fa768f245985 service neutron] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:19:49.406537 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1458/2900] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 11:19:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 11:19:50.270313 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aab07b2-eaaa-491c-9ade-0ed5abc3f1dd service placement] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:19:50.273354 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-6aab07b2-eaaa-491c-9ade-0ed5abc3f1dd service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:19:50.293756 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aab07b2-eaaa-491c-9ade-0ed5abc3f1dd service placement] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:19:50.299369 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ba9006-1025-42e5-8723-ae90312a82e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:19:50.300002 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ba9006-1025-42e5-8723-ae90312a82e0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:19:50.300282 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ba9006-1025-42e5-8723-ae90312a82e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:19:50.300553 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ba9006-1025-42e5-8723-ae90312a82e0 admin admin] PATH_INFO: `/v3/users/93bec07110aa4697ab59ced454a51f9a` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:19:50.301697 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ba9006-1025-42e5-8723-ae90312a82e0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=93bec07110aa4697ab59ced454a51f9a)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:19:50.303640 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ba9006-1025-42e5-8723-ae90312a82e0 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:19:50.366284 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-f9ba9006-1025-42e5-8723-ae90312a82e0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '93bec07110aa4697ab59ced454a51f9a'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:19:50.374402 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-f9ba9006-1025-42e5-8723-ae90312a82e0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '93bec07110aa4697ab59ced454a51f9a'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:19:50.375335 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-f9ba9006-1025-42e5-8723-ae90312a82e0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '93bec07110aa4697ab59ced454a51f9a'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:19:50.391751 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-f9ba9006-1025-42e5-8723-ae90312a82e0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '93bec07110aa4697ab59ced454a51f9a'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:19:50.392590 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-f9ba9006-1025-42e5-8723-ae90312a82e0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '93bec07110aa4697ab59ced454a51f9a'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:19:50.405290 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1443/2901] 10.222.0.40 () {58 vars in 1224 bytes} [Fri Aug 26 11:19:50 2022] DELETE /identity/v3/users/93bec07110aa4697ab59ced454a51f9a => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 11:19:50.580591 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10739778-6477-4238-9c5f-fa768f245985 service neutron] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:19:50.596310 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10739778-6477-4238-9c5f-fa768f245985 service neutron] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:19:50.656679 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67cf725e-c759-4122-ac41-ffebf5a00ebf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:19:50.657312 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67cf725e-c759-4122-ac41-ffebf5a00ebf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:19:50.657588 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67cf725e-c759-4122-ac41-ffebf5a00ebf admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:19:50.657857 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67cf725e-c759-4122-ac41-ffebf5a00ebf admin admin] PATH_INFO: `/v3/projects/3d84157876f945099fda5fcb9d3f6c20` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:19:50.659011 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67cf725e-c759-4122-ac41-ffebf5a00ebf admin admin] RBAC: Authorizing `identity:delete_project(project_id=3d84157876f945099fda5fcb9d3f6c20)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:19:50.660891 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67cf725e-c759-4122-ac41-ffebf5a00ebf admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:19:50.711531 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-67cf725e-c759-4122-ac41-ffebf5a00ebf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3d84157876f945099fda5fcb9d3f6c20', 'request_id': 'req-67cf725e-c759-4122-ac41-ffebf5a00ebf'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:19:50.718196 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-67cf725e-c759-4122-ac41-ffebf5a00ebf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3d84157876f945099fda5fcb9d3f6c20', 'request_id': 'req-67cf725e-c759-4122-ac41-ffebf5a00ebf'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:19:50.725417 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-67cf725e-c759-4122-ac41-ffebf5a00ebf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3d84157876f945099fda5fcb9d3f6c20', 'request_id': 'req-67cf725e-c759-4122-ac41-ffebf5a00ebf'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:19:50.741869 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-67cf725e-c759-4122-ac41-ffebf5a00ebf admin admin] The token cache is being invalidate because project 3d84157876f945099fda5fcb9d3f6c20 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81424) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 11:19:50.742253 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-67cf725e-c759-4122-ac41-ffebf5a00ebf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:19:50.742955 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-67cf725e-c759-4122-ac41-ffebf5a00ebf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:19:50.744749 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1459/2902] 10.222.0.40 () {58 vars in 1233 bytes} [Fri Aug 26 11:19:50 2022] DELETE /identity/v3/projects/3d84157876f945099fda5fcb9d3f6c20 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 11:19:50.749252 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ba9006-1025-42e5-8723-ae90312a82e0 admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:19:50.751942 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-f9ba9006-1025-42e5-8723-ae90312a82e0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:19:50.767023 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ba9006-1025-42e5-8723-ae90312a82e0 admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:19:50.825935 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f754afcb-105a-47ad-a9c7-f1b0578731af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:19:50.826595 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f754afcb-105a-47ad-a9c7-f1b0578731af admin admin] REQUEST_METHOD: `DELETE` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:19:50.826865 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f754afcb-105a-47ad-a9c7-f1b0578731af admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:19:50.827132 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f754afcb-105a-47ad-a9c7-f1b0578731af admin admin] PATH_INFO: `/v3/users/c5f437d3ad9847069fda5361c153cd6c` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:19:50.828287 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f754afcb-105a-47ad-a9c7-f1b0578731af admin admin] RBAC: Authorizing `identity:delete_user(user_id=c5f437d3ad9847069fda5361c153cd6c)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:19:50.830481 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f754afcb-105a-47ad-a9c7-f1b0578731af admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:19:50.833760 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 11:19:50.881160 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-f754afcb-105a-47ad-a9c7-f1b0578731af admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c5f437d3ad9847069fda5361c153cd6c'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:19:50.888095 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-f754afcb-105a-47ad-a9c7-f1b0578731af admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c5f437d3ad9847069fda5361c153cd6c'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:19:50.888765 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-f754afcb-105a-47ad-a9c7-f1b0578731af admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c5f437d3ad9847069fda5361c153cd6c'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:19:50.900598 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-f754afcb-105a-47ad-a9c7-f1b0578731af admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c5f437d3ad9847069fda5361c153cd6c'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:19:50.901261 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-f754afcb-105a-47ad-a9c7-f1b0578731af admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c5f437d3ad9847069fda5361c153cd6c'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:19:50.914254 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1444/2903] 10.222.0.40 () {58 vars in 1224 bytes} [Fri Aug 26 11:19:50 2022] DELETE /identity/v3/users/c5f437d3ad9847069fda5361c153cd6c => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 11:19:51.100134 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67cf725e-c759-4122-ac41-ffebf5a00ebf admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:19:51.101730 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-67cf725e-c759-4122-ac41-ffebf5a00ebf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:19:51.119875 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67cf725e-c759-4122-ac41-ffebf5a00ebf admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:19:51.145100 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 11:19:51.188340 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae07148c-9aa9-411d-8e1f-5f3d23591537 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca155a48e2024c32af3fd2e19e5fac11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3942b05c97e4a1ca048ce3a0379b7a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:19:51.188887 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae07148c-9aa9-411d-8e1f-5f3d23591537 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:19:51.189163 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae07148c-9aa9-411d-8e1f-5f3d23591537 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:19:51.189429 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae07148c-9aa9-411d-8e1f-5f3d23591537 admin admin] PATH_INFO: `/v3/projects/3b7d623925884b588481227e89255494` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:19:51.190946 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae07148c-9aa9-411d-8e1f-5f3d23591537 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3b7d623925884b588481227e89255494)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:19:51.192894 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae07148c-9aa9-411d-8e1f-5f3d23591537 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:19:51.243300 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-ae07148c-9aa9-411d-8e1f-5f3d23591537 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3b7d623925884b588481227e89255494', 'request_id': 'req-ae07148c-9aa9-411d-8e1f-5f3d23591537'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:19:51.252632 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-ae07148c-9aa9-411d-8e1f-5f3d23591537 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3b7d623925884b588481227e89255494', 'request_id': 'req-ae07148c-9aa9-411d-8e1f-5f3d23591537'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:19:51.259581 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-ae07148c-9aa9-411d-8e1f-5f3d23591537 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3b7d623925884b588481227e89255494', 'request_id': 'req-ae07148c-9aa9-411d-8e1f-5f3d23591537'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:19:51.275021 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-ae07148c-9aa9-411d-8e1f-5f3d23591537 admin admin] The token cache is being invalidate because project 3b7d623925884b588481227e89255494 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81424) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 11:19:51.275389 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-ae07148c-9aa9-411d-8e1f-5f3d23591537 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:19:51.276192 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-ae07148c-9aa9-411d-8e1f-5f3d23591537 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 11:19:51.278111 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1460/2904] 10.222.0.40 () {58 vars in 1233 bytes} [Fri Aug 26 11:19:51 2022] DELETE /identity/v3/projects/3b7d623925884b588481227e89255494 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 11:20:00.910769 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=25,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 11:20:01.271212 dv-cinder-852369-2-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=40,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 11:23:00.752570 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f754afcb-105a-47ad-a9c7-f1b0578731af admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 11:23:00.754453 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-f754afcb-105a-47ad-a9c7-f1b0578731af admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:23:00.925939 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f754afcb-105a-47ad-a9c7-f1b0578731af admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 11:23:00.988413 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ea52b0-2b57-48d5-b4d7-3767110ee314 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f29c5dfed31f42a9b5cbc0cd72e9fdc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e30ccf0c1204727b786876c141dbcd2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 11:23:00.989069 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ea52b0-2b57-48d5-b4d7-3767110ee314 service placement] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 11:23:00.989386 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ea52b0-2b57-48d5-b4d7-3767110ee314 service placement] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 11:23:00.989662 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ea52b0-2b57-48d5-b4d7-3767110ee314 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 11:23:00.991602 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-66ea52b0-2b57-48d5-b4d7-3767110ee314 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 11:23:01.003924 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ea52b0-2b57-48d5-b4d7-3767110ee314 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 11:23:01.005424 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ea52b0-2b57-48d5-b4d7-3767110ee314 service placement] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 11:23:01.016034 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1445/2905] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 11:23:00 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 264 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 11:23:11.011362 dv-cinder-852369-2-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=89 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}